Add files using upload-large-folder tool

runs/fixed_internvl/context/Deep8_s13/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Deep8_s13/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.28.mlp.down_proj": 144,
+    "model.language_model.layers.28.mlp.gate_proj": 144,
+    "model.language_model.layers.28.mlp.up_proj": 144,
+    "model.language_model.layers.28.self_attn.k_proj": 144,
+    "model.language_model.layers.28.self_attn.o_proj": 144,
+    "model.language_model.layers.28.self_attn.q_proj": 144,
+    "model.language_model.layers.28.self_attn.v_proj": 144,
+    "model.language_model.layers.29.mlp.down_proj": 144,
+    "model.language_model.layers.29.mlp.gate_proj": 144,
+    "model.language_model.layers.29.mlp.up_proj": 144,
+    "model.language_model.layers.29.self_attn.k_proj": 144,
+    "model.language_model.layers.29.self_attn.o_proj": 144,
+    "model.language_model.layers.29.self_attn.q_proj": 144,
+    "model.language_model.layers.29.self_attn.v_proj": 144,
+    "model.language_model.layers.30.mlp.down_proj": 144,
+    "model.language_model.layers.30.mlp.gate_proj": 144,
+    "model.language_model.layers.30.mlp.up_proj": 144,
+    "model.language_model.layers.30.self_attn.k_proj": 144,
+    "model.language_model.layers.30.self_attn.o_proj": 144,
+    "model.language_model.layers.30.self_attn.q_proj": 144,
+    "model.language_model.layers.30.self_attn.v_proj": 144,
+    "model.language_model.layers.31.mlp.down_proj": 144,
+    "model.language_model.layers.31.mlp.gate_proj": 144,
+    "model.language_model.layers.31.mlp.up_proj": 144,
+    "model.language_model.layers.31.self_attn.k_proj": 144,
+    "model.language_model.layers.31.self_attn.o_proj": 144,
+    "model.language_model.layers.31.self_attn.q_proj": 144,
+    "model.language_model.layers.31.self_attn.v_proj": 144,
+    "model.language_model.layers.32.mlp.down_proj": 144,
+    "model.language_model.layers.32.mlp.gate_proj": 144,
+    "model.language_model.layers.32.mlp.up_proj": 144,
+    "model.language_model.layers.32.self_attn.k_proj": 144,
+    "model.language_model.layers.32.self_attn.o_proj": 144,
+    "model.language_model.layers.32.self_attn.q_proj": 144,
+    "model.language_model.layers.32.self_attn.v_proj": 144,
+    "model.language_model.layers.33.mlp.down_proj": 144,
+    "model.language_model.layers.33.mlp.gate_proj": 144,
+    "model.language_model.layers.33.mlp.up_proj": 144,
+    "model.language_model.layers.33.self_attn.k_proj": 144,
+    "model.language_model.layers.33.self_attn.o_proj": 144,
+    "model.language_model.layers.33.self_attn.q_proj": 144,
+    "model.language_model.layers.33.self_attn.v_proj": 144,
+    "model.language_model.layers.34.mlp.down_proj": 144,
+    "model.language_model.layers.34.mlp.gate_proj": 144,
+    "model.language_model.layers.34.mlp.up_proj": 144,
+    "model.language_model.layers.34.self_attn.k_proj": 144,
+    "model.language_model.layers.34.self_attn.o_proj": 144,
+    "model.language_model.layers.34.self_attn.q_proj": 144,
+    "model.language_model.layers.34.self_attn.v_proj": 144,
+    "model.language_model.layers.35.mlp.down_proj": 144,
+    "model.language_model.layers.35.mlp.gate_proj": 144,
+    "model.language_model.layers.35.mlp.up_proj": 144,
+    "model.language_model.layers.35.self_attn.k_proj": 144,
+    "model.language_model.layers.35.self_attn.o_proj": 144,
+    "model.language_model.layers.35.self_attn.q_proj": 144,
+    "model.language_model.layers.35.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.28.mlp.down_proj": 72,
+    "model.language_model.layers.28.mlp.gate_proj": 72,
+    "model.language_model.layers.28.mlp.up_proj": 72,
+    "model.language_model.layers.28.self_attn.k_proj": 72,
+    "model.language_model.layers.28.self_attn.o_proj": 72,
+    "model.language_model.layers.28.self_attn.q_proj": 72,
+    "model.language_model.layers.28.self_attn.v_proj": 72,
+    "model.language_model.layers.29.mlp.down_proj": 72,
+    "model.language_model.layers.29.mlp.gate_proj": 72,
+    "model.language_model.layers.29.mlp.up_proj": 72,
+    "model.language_model.layers.29.self_attn.k_proj": 72,
+    "model.language_model.layers.29.self_attn.o_proj": 72,
+    "model.language_model.layers.29.self_attn.q_proj": 72,
+    "model.language_model.layers.29.self_attn.v_proj": 72,
+    "model.language_model.layers.30.mlp.down_proj": 72,
+    "model.language_model.layers.30.mlp.gate_proj": 72,
+    "model.language_model.layers.30.mlp.up_proj": 72,
+    "model.language_model.layers.30.self_attn.k_proj": 72,
+    "model.language_model.layers.30.self_attn.o_proj": 72,
+    "model.language_model.layers.30.self_attn.q_proj": 72,
+    "model.language_model.layers.30.self_attn.v_proj": 72,
+    "model.language_model.layers.31.mlp.down_proj": 72,
+    "model.language_model.layers.31.mlp.gate_proj": 72,
+    "model.language_model.layers.31.mlp.up_proj": 72,
+    "model.language_model.layers.31.self_attn.k_proj": 72,
+    "model.language_model.layers.31.self_attn.o_proj": 72,
+    "model.language_model.layers.31.self_attn.q_proj": 72,
+    "model.language_model.layers.31.self_attn.v_proj": 72,
+    "model.language_model.layers.32.mlp.down_proj": 72,
+    "model.language_model.layers.32.mlp.gate_proj": 72,
+    "model.language_model.layers.32.mlp.up_proj": 72,
+    "model.language_model.layers.32.self_attn.k_proj": 72,
+    "model.language_model.layers.32.self_attn.o_proj": 72,
+    "model.language_model.layers.32.self_attn.q_proj": 72,
+    "model.language_model.layers.32.self_attn.v_proj": 72,
+    "model.language_model.layers.33.mlp.down_proj": 72,
+    "model.language_model.layers.33.mlp.gate_proj": 72,
+    "model.language_model.layers.33.mlp.up_proj": 72,
+    "model.language_model.layers.33.self_attn.k_proj": 72,
+    "model.language_model.layers.33.self_attn.o_proj": 72,
+    "model.language_model.layers.33.self_attn.q_proj": 72,
+    "model.language_model.layers.33.self_attn.v_proj": 72,
+    "model.language_model.layers.34.mlp.down_proj": 72,
+    "model.language_model.layers.34.mlp.gate_proj": 72,
+    "model.language_model.layers.34.mlp.up_proj": 72,
+    "model.language_model.layers.34.self_attn.k_proj": 72,
+    "model.language_model.layers.34.self_attn.o_proj": 72,
+    "model.language_model.layers.34.self_attn.q_proj": 72,
+    "model.language_model.layers.34.self_attn.v_proj": 72,
+    "model.language_model.layers.35.mlp.down_proj": 72,
+    "model.language_model.layers.35.mlp.gate_proj": 72,
+    "model.language_model.layers.35.mlp.up_proj": 72,
+    "model.language_model.layers.35.self_attn.k_proj": 72,
+    "model.language_model.layers.35.self_attn.o_proj": 72,
+    "model.language_model.layers.35.self_attn.q_proj": 72,
+    "model.language_model.layers.35.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.32.self_attn.v_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.29.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.k_proj",
+    "model.language_model.layers.35.self_attn.k_proj",
+    "model.language_model.layers.31.mlp.up_proj",
+    "model.language_model.layers.33.mlp.gate_proj",
+    "model.language_model.layers.30.self_attn.q_proj",
+    "model.language_model.layers.30.self_attn.k_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.33.self_attn.v_proj",
+    "model.language_model.layers.30.self_attn.v_proj",
+    "model.language_model.layers.30.mlp.up_proj",
+    "model.language_model.layers.31.self_attn.q_proj",
+    "model.language_model.layers.30.mlp.gate_proj",
+    "model.language_model.layers.32.mlp.up_proj",
+    "model.language_model.layers.28.mlp.gate_proj",
+    "model.language_model.layers.32.self_attn.o_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.33.self_attn.o_proj",
+    "model.language_model.layers.33.self_attn.k_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.31.mlp.gate_proj",
+    "model.language_model.layers.34.mlp.down_proj",
+    "model.language_model.layers.32.mlp.gate_proj",
+    "model.language_model.layers.33.self_attn.q_proj",
+    "model.language_model.layers.32.self_attn.k_proj",
+    "model.language_model.layers.35.self_attn.o_proj",
+    "model.language_model.layers.31.mlp.down_proj",
+    "model.language_model.layers.30.self_attn.o_proj",
+    "model.language_model.layers.33.mlp.down_proj",
+    "model.language_model.layers.35.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.31.self_attn.o_proj",
+    "model.language_model.layers.31.self_attn.k_proj",
+    "model.language_model.layers.28.self_attn.v_proj",
+    "model.language_model.layers.29.self_attn.k_proj",
+    "model.language_model.layers.30.mlp.down_proj",
+    "model.language_model.layers.34.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.down_proj",
+    "model.language_model.layers.34.mlp.up_proj",
+    "model.language_model.layers.35.mlp.down_proj",
+    "model.language_model.layers.33.mlp.up_proj",
+    "model.language_model.layers.35.mlp.gate_proj",
+    "model.language_model.layers.32.mlp.down_proj",
+    "model.language_model.layers.34.mlp.gate_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.31.self_attn.v_proj",
+    "model.language_model.layers.34.self_attn.q_proj",
+    "model.language_model.layers.35.mlp.up_proj",
+    "model.language_model.layers.32.self_attn.q_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.34.self_attn.k_proj",
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.35.self_attn.q_proj",
+    "model.language_model.layers.34.self_attn.o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Deep8_s13/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 13,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Deep8",
+    "layers": [
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35
+    ],
+    "ranks": {
+      "28": 72,
+      "29": 72,
+      "30": 72,
+      "31": 72,
+      "32": 72,
+      "33": 72,
+      "34": 72,
+      "35": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Deep8_s13/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 1.812181830406189, "loss_scale": 4.0, "retries": 0, "seconds": 2.2, "max_memory_gb": 16.82, "ce": 0.5227766726748087, "kl": 0.08654405153356493, "context": 0.5660487064160407}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 2.5002501010894775, "loss_scale": 4.0, "retries": 0, "seconds": 3.6, "max_memory_gb": 17.16, "ce": 0.6370583198950044, "kl": 0.4672029710179686, "context": 0.8706598142525763}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 5.444128036499023, "loss_scale": 4.0, "retries": 0, "seconds": 4.8, "max_memory_gb": 17.16, "ce": 1.0801377519965172, "kl": 0.06854924000799656, "context": 1.114412359893322}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 2.9628217220306396, "loss_scale": 4.0, "retries": 0, "seconds": 6.1, "max_memory_gb": 17.16, "ce": 0.4243074264668394, "kl": 0.4303124627288071, "context": 0.6394636367040221}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 0.07388123124837875, "loss_scale": 4.0, "retries": 0, "seconds": 7.2, "max_memory_gb": 17.16, "ce": 0.009455782290388015, "kl": 0.008399454643949866, "context": 0.013655509290401824}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 1.5715748071670532, "loss_scale": 4.0, "retries": 0, "seconds": 8.4, "max_memory_gb": 17.16, "ce": 0.2512020075228065, "kl": 0.07727284164502635, "context": 0.2898384299769532}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 4.149362087249756, "loss_scale": 4.0, "retries": 0, "seconds": 9.7, "max_memory_gb": 17.16, "ce": 0.6523881078956038, "kl": 0.13780375702981473, "context": 0.7212899907171959}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 4.742311000823975, "loss_scale": 4.0, "retries": 0, "seconds": 10.9, "max_memory_gb": 17.16, "ce": 0.5290221078321338, "kl": 0.27690887823700905, "context": 0.6674765534698963}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 2.94350528717041, "loss_scale": 4.0, "retries": 0, "seconds": 11.9, "max_memory_gb": 17.16, "ce": 0.32971244119107723, "kl": 0.14058688236400485, "context": 0.40000587701797485}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 0.36275264620780945, "loss_scale": 4.0, "retries": 0, "seconds": 13.2, "max_memory_gb": 17.17, "ce": 0.01835650607245043, "kl": 0.03650940035004169, "context": 0.03661120717879385}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 2.9789609909057617, "loss_scale": 4.0, "retries": 0, "seconds": 14.5, "max_memory_gb": 17.17, "ce": 0.26278729269688483, "kl": 0.006145508888494078, "context": 0.2658600529030082}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 1.7524590492248535, "loss_scale": 4.0, "retries": 0, "seconds": 15.7, "max_memory_gb": 17.17, "ce": 0.13266762145212851, "kl": 0.00769646729941087, "context": 0.1365158588396298}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 1.277373194694519, "loss_scale": 4.0, "retries": 0, "seconds": 17.0, "max_memory_gb": 17.17, "ce": 0.07405381870921701, "kl": 0.004092818877325044, "context": 0.07610022867447697}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 2.6385324001312256, "loss_scale": 4.0, "retries": 0, "seconds": 18.3, "max_memory_gb": 17.17, "ce": 0.14630828518420458, "kl": 0.7798234634101391, "context": 0.5362200140953064}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 1.7272673845291138, "loss_scale": 4.0, "retries": 0, "seconds": 19.6, "max_memory_gb": 17.17, "ce": 0.0305775954134333, "kl": 0.35678735339911327, "context": 0.2089712662996135}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 0.07364799827337265, "loss_scale": 4.0, "retries": 0, "seconds": 20.8, "max_memory_gb": 17.17, "ce": 0.004997973821446067, "kl": 0.0009932242837749072, "context": 0.005494586112035904}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 1.1263378858566284, "loss_scale": 4.0, "retries": 0, "seconds": 21.9, "max_memory_gb": 17.17, "ce": 0.07678368443157524, "kl": 0.771829182820511, "context": 0.4626982837507967}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 1.1623005867004395, "loss_scale": 4.0, "retries": 0, "seconds": 23.2, "max_memory_gb": 17.17, "ce": 0.005871358283911832, "kl": 0.19301155040011508, "context": 0.10237713038804941}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.5272781252861023, "loss_scale": 4.0, "retries": 0, "seconds": 24.5, "max_memory_gb": 17.17, "ce": 0.007640933901711833, "kl": 0.061808698297681985, "context": 0.038545283787243534}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.12629635632038116, "loss_scale": 4.0, "retries": 0, "seconds": 25.6, "max_memory_gb": 17.17, "ce": 0.0017145065212389454, "kl": 0.023422316859068815, "context": 0.013425664605165366}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.028014322742819786, "loss_scale": 4.0, "retries": 0, "seconds": 27.0, "max_memory_gb": 17.17, "ce": 0.0006025260736350901, "kl": 0.0016008519114620867, "context": 0.0014029520589247113}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 2.3166513442993164, "loss_scale": 4.0, "retries": 0, "seconds": 28.2, "max_memory_gb": 17.17, "ce": 0.07420578552410007, "kl": 0.06320566335489275, "context": 0.10580861626658589}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 4.8444037437438965, "loss_scale": 4.0, "retries": 0, "seconds": 29.3, "max_memory_gb": 17.17, "ce": 0.053133628565774416, "kl": 0.3104921901685884, "context": 0.20837972634217294}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.4193844497203827, "loss_scale": 4.0, "retries": 0, "seconds": 30.6, "max_memory_gb": 17.17, "ce": 0.006335942831356078, "kl": 0.02312395744229434, "context": 0.017897921192343347}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 3.7011940479278564, "loss_scale": 4.0, "retries": 0, "seconds": 31.8, "max_memory_gb": 17.17, "ce": 0.1673484880302567, "kl": 0.036944972118362784, "context": 0.18582098092883825}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 1.9954259395599365, "loss_scale": 4.0, "retries": 0, "seconds": 33.2, "max_memory_gb": 17.17, "ce": 0.05375188577454537, "kl": 0.5376869943756901, "context": 0.3225953792934888}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 1.5959731340408325, "loss_scale": 4.0, "retries": 0, "seconds": 34.3, "max_memory_gb": 17.17, "ce": 0.007506129921239335, "kl": 0.23210993677548686, "context": 0.12356109911343083}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 2.327549934387207, "loss_scale": 4.0, "retries": 0, "seconds": 35.3, "max_memory_gb": 17.17, "ce": 0.28955295400260184, "kl": 0.24374229926615953, "context": 0.41142410784959793}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 1.9216989278793335, "loss_scale": 4.0, "retries": 0, "seconds": 36.4, "max_memory_gb": 17.17, "ce": 0.23438446265936363, "kl": 0.1377454483517795, "context": 0.30325719009852037}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.8041544556617737, "loss_scale": 4.0, "retries": 0, "seconds": 37.5, "max_memory_gb": 17.17, "ce": 0.009571633945597569, "kl": 0.02701514815998962, "context": 0.023079208338458557}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 2.2329540252685547, "loss_scale": 4.0, "retries": 0, "seconds": 38.5, "max_memory_gb": 17.17, "ce": 0.2731182707939297, "kl": 0.07294665742665529, "context": 0.30959160439670086}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 1.474756121635437, "loss_scale": 4.0, "retries": 0, "seconds": 39.6, "max_memory_gb": 17.17, "ce": 0.033845352358184755, "kl": 0.06706631847191602, "context": 0.06737851339858025}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 2.0575454235076904, "loss_scale": 4.0, "retries": 0, "seconds": 40.7, "max_memory_gb": 17.17, "ce": 0.166953457868658, "kl": 0.09119154687505215, "context": 0.21254922938533127}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.4579172134399414, "loss_scale": 4.0, "retries": 0, "seconds": 41.9, "max_memory_gb": 17.17, "ce": 0.005521952785784379, "kl": 0.8125344407817465, "context": 0.41178916351054795}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.039288658648729324, "loss_scale": 4.0, "retries": 0, "seconds": 43.2, "max_memory_gb": 17.17, "ce": 0.0021605125730275176, "kl": 0.0009677864727564156, "context": 0.002644405889441259}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 4.2995381355285645, "loss_scale": 4.0, "retries": 0, "seconds": 44.3, "max_memory_gb": 17.17, "ce": 0.3138902176215197, "kl": 0.01134191112487315, "context": 0.3195611623596051}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 2.6376709938049316, "loss_scale": 4.0, "retries": 0, "seconds": 45.4, "max_memory_gb": 17.17, "ce": 0.09967949890415184, "kl": 0.7331476365798153, "context": 0.4662533125374466}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 2.4344699382781982, "loss_scale": 4.0, "retries": 0, "seconds": 46.6, "max_memory_gb": 17.17, "ce": 0.10513919993536547, "kl": 0.43449529476311, "context": 0.32238684309413657}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 0.7039386034011841, "loss_scale": 4.0, "retries": 0, "seconds": 47.8, "max_memory_gb": 17.17, "ce": 0.10896424192833365, "kl": 0.008893082602298819, "context": 0.11341078365512658}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.12234393507242203, "loss_scale": 4.0, "retries": 0, "seconds": 49.1, "max_memory_gb": 17.17, "ce": 0.005200468527618796, "kl": 0.006076518853660673, "context": 0.008238727925345302}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 1.5136289596557617, "loss_scale": 4.0, "retries": 0, "seconds": 50.4, "max_memory_gb": 17.17, "ce": 0.3704541700426489, "kl": 0.10028676502406597, "context": 0.42059754487127066}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 0.0672064796090126, "loss_scale": 4.0, "retries": 0, "seconds": 51.6, "max_memory_gb": 17.17, "ce": 0.003102694929111749, "kl": 0.0003326835358166136, "context": 0.003269036766141653}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 0.2903884947299957, "loss_scale": 4.0, "retries": 0, "seconds": 52.9, "max_memory_gb": 17.17, "ce": 0.009289817520766519, "kl": 0.005105824617203325, "context": 0.011842729581985623}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 3.3040456771850586, "loss_scale": 4.0, "retries": 0, "seconds": 54.1, "max_memory_gb": 17.17, "ce": 0.35967659691232257, "kl": 1.0860589302756125, "context": 0.9027060579392128}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 1.9447078704833984, "loss_scale": 4.0, "retries": 0, "seconds": 55.2, "max_memory_gb": 17.17, "ce": 0.4801703990669921, "kl": 0.14586229812994134, "context": 0.5531015625456348}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 0.7393617033958435, "loss_scale": 4.0, "retries": 0, "seconds": 56.5, "max_memory_gb": 17.17, "ce": 0.08116464741760865, "kl": 0.15267023828346282, "context": 0.1574997646966949}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 1.0881098508834839, "loss_scale": 4.0, "retries": 0, "seconds": 57.5, "max_memory_gb": 17.17, "ce": 0.023649929091334343, "kl": 0.08873500076879282, "context": 0.06801743136020377}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.03351658582687378, "loss_scale": 4.0, "retries": 0, "seconds": 58.8, "max_memory_gb": 17.17, "ce": 0.0007536836856161244, "kl": 0.005150898801730364, "context": 0.003329132989165373}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.7504462003707886, "loss_scale": 4.0, "retries": 0, "seconds": 60.0, "max_memory_gb": 17.17, "ce": 0.0105929309502244, "kl": 0.07232319558784184, "context": 0.04675452839001082}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 1.2799723148345947, "loss_scale": 4.0, "retries": 0, "seconds": 61.1, "max_memory_gb": 17.17, "ce": 0.059000132969231345, "kl": 0.008226783320424147, "context": 0.06311352463671938}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.2174360752105713, "loss_scale": 4.0, "retries": 0, "seconds": 62.5, "max_memory_gb": 17.17, "ce": 0.1108363901003031, "kl": 0.012225408093968326, "context": 0.11694909035577439}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.7817173004150391, "loss_scale": 4.0, "retries": 0, "seconds": 63.7, "max_memory_gb": 17.17, "ce": 0.012980813859030604, "kl": 0.1270643518655561, "context": 0.07651298749260604}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 0.6972322463989258, "loss_scale": 4.0, "retries": 0, "seconds": 64.9, "max_memory_gb": 17.17, "ce": 0.07240663905395195, "kl": 0.09230654942803085, "context": 0.11855991370975971}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 2.6772079467773438, "loss_scale": 4.0, "retries": 0, "seconds": 66.1, "max_memory_gb": 17.17, "ce": 0.1100555745215388, "kl": 0.1665953122319479, "context": 0.19335323809355032}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 0.1448521465063095, "loss_scale": 4.0, "retries": 0, "seconds": 67.3, "max_memory_gb": 17.17, "ce": 0.008928472816478461, "kl": 0.003905286308054201, "context": 0.010881116322707385}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 1.4168004989624023, "loss_scale": 4.0, "retries": 0, "seconds": 68.5, "max_memory_gb": 17.17, "ce": 0.043971930630505085, "kl": 0.1021740113501437, "context": 0.09505893406458199}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.4447759985923767, "loss_scale": 4.0, "retries": 0, "seconds": 69.9, "max_memory_gb": 17.17, "ce": 0.01058391624246724, "kl": 0.015271444812242407, "context": 0.018219638615846634}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.024496236816048622, "loss_scale": 4.0, "retries": 0, "seconds": 71.1, "max_memory_gb": 17.17, "ce": 0.0015277589918696322, "kl": 0.0008489364673778255, "context": 0.001952227219589986}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.9706892967224121, "loss_scale": 4.0, "retries": 0, "seconds": 72.5, "max_memory_gb": 17.17, "ce": 0.09226395527366549, "kl": 0.03796625856193714, "context": 0.11124708643183112}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 1.9312384128570557, "loss_scale": 4.0, "retries": 0, "seconds": 73.7, "max_memory_gb": 17.17, "ce": 0.08708509616553783, "kl": 0.07066301739541814, "context": 0.12241660105064511}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 2.55452036857605, "loss_scale": 4.0, "retries": 0, "seconds": 75.0, "max_memory_gb": 17.17, "ce": 0.04822125763166696, "kl": 0.5934425515297335, "context": 0.3449425370199606}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.8972410559654236, "loss_scale": 4.0, "retries": 0, "seconds": 76.3, "max_memory_gb": 17.17, "ce": 0.11519795807544142, "kl": 0.012541151176264975, "context": 0.121468535537133}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 1.0026606321334839, "loss_scale": 4.0, "retries": 0, "seconds": 77.6, "max_memory_gb": 17.17, "ce": 0.03640467408695258, "kl": 0.04009341957862489, "context": 0.05645138374529779}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.3242029845714569, "loss_scale": 4.0, "retries": 0, "seconds": 78.9, "max_memory_gb": 17.17, "ce": 0.015917495999019593, "kl": 0.009568590496201068, "context": 0.0207017915090546}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 1.1142122745513916, "loss_scale": 4.0, "retries": 0, "seconds": 80.2, "max_memory_gb": 17.17, "ce": 0.15551569312810898, "kl": 0.11613433994352818, "context": 0.21358286775648594}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 2.3280110359191895, "loss_scale": 4.0, "retries": 0, "seconds": 81.5, "max_memory_gb": 17.17, "ce": 0.140234782641528, "kl": 0.0794852754315798, "context": 0.1799774240971601}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.09067615121603012, "loss_scale": 4.0, "retries": 0, "seconds": 82.7, "max_memory_gb": 17.17, "ce": 0.005124494316987693, "kl": 0.007445332417773898, "context": 0.008847160817822441}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.27270621061325073, "loss_scale": 4.0, "retries": 0, "seconds": 84.0, "max_memory_gb": 17.17, "ce": 0.021264699062157888, "kl": 0.020490024748141877, "context": 0.03150971050490625}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 1.9228219985961914, "loss_scale": 4.0, "retries": 0, "seconds": 85.2, "max_memory_gb": 17.17, "ce": 0.47014783578924835, "kl": 0.017636343283811584, "context": 0.4789660186506808}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.19388391077518463, "loss_scale": 4.0, "retries": 0, "seconds": 86.5, "max_memory_gb": 17.17, "ce": 0.012885233736597002, "kl": 0.0008492928336636396, "context": 0.013309879825101234}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.8915916085243225, "loss_scale": 4.0, "retries": 0, "seconds": 87.7, "max_memory_gb": 17.17, "ce": 0.02948242365164333, "kl": 0.012287422412100568, "context": 0.03562613577742013}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 1.2359211444854736, "loss_scale": 4.0, "retries": 0, "seconds": 88.9, "max_memory_gb": 17.17, "ce": 0.022830390837043524, "kl": 0.4875044960062951, "context": 0.26658263336867094}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 0.850096583366394, "loss_scale": 4.0, "retries": 0, "seconds": 90.2, "max_memory_gb": 17.17, "ce": 0.15565147854795214, "kl": 0.07432271524157841, "context": 0.19281282869633287}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.4960964024066925, "loss_scale": 4.0, "retries": 0, "seconds": 91.4, "max_memory_gb": 17.17, "ce": 0.1237428292442928, "kl": 0.044147657325083856, "context": 0.14581665790683473}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.4971617460250854, "loss_scale": 4.0, "retries": 0, "seconds": 92.8, "max_memory_gb": 17.17, "ce": 0.5366569122998044, "kl": 0.788906242698431, "context": 0.931110018864274}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 2.411860942840576, "loss_scale": 4.0, "retries": 0, "seconds": 94.1, "max_memory_gb": 17.17, "ce": 0.008811871928628534, "kl": 1.4424456903943792, "context": 0.7300347306299955}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.3461388051509857, "loss_scale": 4.0, "retries": 0, "seconds": 95.3, "max_memory_gb": 17.17, "ce": 0.014135750418063253, "kl": 0.0067808184739988064, "context": 0.01752615999430418}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.7102403044700623, "loss_scale": 4.0, "retries": 0, "seconds": 96.5, "max_memory_gb": 17.17, "ce": 0.002365042208111845, "kl": 0.07489756667928305, "context": 0.03981382661731914}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.26708441972732544, "loss_scale": 4.0, "retries": 0, "seconds": 97.8, "max_memory_gb": 17.17, "ce": 0.02111745684305788, "kl": 0.02754199093396892, "context": 0.03488845237006899}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.47131404280662537, "loss_scale": 4.0, "retries": 0, "seconds": 99.0, "max_memory_gb": 17.17, "ce": 0.0027342792782292236, "kl": 0.03789943751181113, "context": 0.021683997350919526}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 0.8572149872779846, "loss_scale": 4.0, "retries": 0, "seconds": 100.3, "max_memory_gb": 17.17, "ce": 0.07426515300903702, "kl": 0.04305198701786139, "context": 0.09579114694497548}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.09075267612934113, "loss_scale": 4.0, "retries": 0, "seconds": 101.4, "max_memory_gb": 17.17, "ce": 0.009494683530647308, "kl": 0.0019294319627078949, "context": 0.010459399796673097}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 2.1873996257781982, "loss_scale": 4.0, "retries": 0, "seconds": 102.6, "max_memory_gb": 17.17, "ce": 0.07464515666652005, "kl": 0.6297564269116265, "context": 0.38952335905923974}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 3.772671937942505, "loss_scale": 4.0, "retries": 0, "seconds": 103.9, "max_memory_gb": 17.17, "ce": 0.2997529194908566, "kl": 0.06351046054351173, "context": 0.33150815458066063}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.16011419892311096, "loss_scale": 4.0, "retries": 0, "seconds": 105.1, "max_memory_gb": 17.17, "ce": 0.010103080527187558, "kl": 0.001507803912829786, "context": 0.010856982305995189}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.8416686058044434, "loss_scale": 4.0, "retries": 0, "seconds": 106.3, "max_memory_gb": 17.17, "ce": 0.053500251786317676, "kl": 0.1658465339569375, "context": 0.13642351934686303}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 1.4778759479522705, "loss_scale": 4.0, "retries": 0, "seconds": 107.6, "max_memory_gb": 17.17, "ce": 0.004748054489027709, "kl": 0.7324123680882622, "context": 0.3709542391006835}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.28394174575805664, "loss_scale": 4.0, "retries": 0, "seconds": 108.8, "max_memory_gb": 17.17, "ce": 0.027070375261246227, "kl": 0.0036374794699725044, "context": 0.028889115012134425}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.18133105337619781, "loss_scale": 4.0, "retries": 0, "seconds": 109.9, "max_memory_gb": 17.17, "ce": 0.021958471012112568, "kl": 0.0154502781406336, "context": 0.029683610948268324}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.11543586105108261, "loss_scale": 4.0, "retries": 0, "seconds": 111.2, "max_memory_gb": 17.17, "ce": 0.013157008914276958, "kl": 0.002896241843700409, "context": 0.01460512971971184}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 1.626430630683899, "loss_scale": 4.0, "retries": 0, "seconds": 112.4, "max_memory_gb": 17.17, "ce": 0.4189282711377018, "kl": 0.2062638176175824, "context": 0.5220601743640145}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 1.8636984825134277, "loss_scale": 4.0, "retries": 0, "seconds": 113.7, "max_memory_gb": 17.17, "ce": 0.17833327456901316, "kl": 0.30683224193580827, "context": 0.3317493800568627}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 0.9206052422523499, "loss_scale": 4.0, "retries": 0, "seconds": 114.9, "max_memory_gb": 17.17, "ce": 0.05274313386689755, "kl": 0.2642732804597472, "context": 0.18487977038603276}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 0.81131511926651, "loss_scale": 4.0, "retries": 0, "seconds": 116.2, "max_memory_gb": 17.17, "ce": 0.06366272366722114, "kl": 0.019730922911548987, "context": 0.07352818321669474}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 1.7402982711791992, "loss_scale": 4.0, "retries": 0, "seconds": 117.5, "max_memory_gb": 17.17, "ce": 0.39448140573949786, "kl": 0.025226724974345416, "context": 0.4070947610598523}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 1.1202974319458008, "loss_scale": 4.0, "retries": 0, "seconds": 118.8, "max_memory_gb": 17.17, "ce": 0.3094506870547775, "kl": 0.003624161625339184, "context": 0.31126275926362723}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.8359766602516174, "loss_scale": 4.0, "retries": 0, "seconds": 120.0, "max_memory_gb": 17.17, "ce": 0.011010028363671154, "kl": 0.10725601846934296, "context": 0.06463803676888347}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.5974938869476318, "loss_scale": 4.0, "retries": 0, "seconds": 121.3, "max_memory_gb": 17.17, "ce": 0.0016925452873692848, "kl": 0.06818350423418451, "context": 0.03578429740446154}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.16940727829933167, "loss_scale": 4.0, "retries": 0, "seconds": 122.5, "max_memory_gb": 17.17, "ce": 0.020146329174167477, "kl": 0.006431978814362083, "context": 0.023362318199360743}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.8482698798179626, "loss_scale": 4.0, "retries": 0, "seconds": 123.8, "max_memory_gb": 17.17, "ce": 0.03235358005622402, "kl": 0.44509652652050136, "context": 0.25490183680085465}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.30787962675094604, "loss_scale": 4.0, "retries": 0, "seconds": 125.1, "max_memory_gb": 17.17, "ce": 0.004517650661000516, "kl": 0.08284916276898002, "context": 0.04594223134336062}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 1.1024107933044434, "loss_scale": 4.0, "retries": 0, "seconds": 126.3, "max_memory_gb": 17.17, "ce": 0.055466983234509826, "kl": 0.013238392857601866, "context": 0.06208618078380823}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.16102838516235352, "loss_scale": 4.0, "retries": 0, "seconds": 127.6, "max_memory_gb": 17.17, "ce": 0.010310036959708668, "kl": 0.012940934324433329, "context": 0.016780504243797623}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.5252429246902466, "loss_scale": 4.0, "retries": 0, "seconds": 128.7, "max_memory_gb": 17.17, "ce": 0.03796425736072706, "kl": 0.007481925470074202, "context": 0.04170522151252953}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 1.005780816078186, "loss_scale": 4.0, "retries": 0, "seconds": 129.9, "max_memory_gb": 17.17, "ce": 0.026467526506166905, "kl": 0.18014334260078613, "context": 0.1165392006514594}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.05167245492339134, "loss_scale": 4.0, "retries": 0, "seconds": 131.2, "max_memory_gb": 17.17, "ce": 0.003938190573535394, "kl": 0.0006173441615828779, "context": 0.004246862714353483}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.34503838419914246, "loss_scale": 4.0, "retries": 0, "seconds": 132.5, "max_memory_gb": 17.17, "ce": 0.03562348058039788, "kl": 0.1400426699108266, "context": 0.1056448197341524}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 1.5144912004470825, "loss_scale": 4.0, "retries": 0, "seconds": 133.6, "max_memory_gb": 17.17, "ce": 0.25817210515378974, "kl": 0.009350387910103564, "context": 0.2628473096701782}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 1.747531771659851, "loss_scale": 4.0, "retries": 0, "seconds": 134.9, "max_memory_gb": 17.17, "ce": 0.25129214688786305, "kl": 0.18438460887409747, "context": 0.3434844480943866}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.9781686067581177, "loss_scale": 4.0, "retries": 0, "seconds": 136.1, "max_memory_gb": 17.17, "ce": 0.0551076652482152, "kl": 0.027828104561194777, "context": 0.06902171717956662}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.01655208319425583, "loss_scale": 4.0, "retries": 0, "seconds": 137.3, "max_memory_gb": 17.17, "ce": 0.0010810808234964497, "kl": 0.0008955265093391063, "context": 0.0015288441209122539}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.3254707157611847, "loss_scale": 4.0, "retries": 0, "seconds": 138.6, "max_memory_gb": 17.17, "ce": 0.020156158891040832, "kl": 0.002453635214806127, "context": 0.02138297593774041}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 0.44416773319244385, "loss_scale": 4.0, "retries": 0, "seconds": 139.8, "max_memory_gb": 17.17, "ce": 0.02323285659076646, "kl": 0.04859227349516004, "context": 0.047528992872685194}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 2.0881381034851074, "loss_scale": 4.0, "retries": 0, "seconds": 141.0, "max_memory_gb": 17.17, "ce": 0.012674357763899025, "kl": 0.40684174411944696, "context": 0.21609523046936374}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 0.7135800123214722, "loss_scale": 4.0, "retries": 0, "seconds": 142.2, "max_memory_gb": 17.17, "ce": 0.04778380028437823, "kl": 0.040015062266320456, "context": 0.06779133132658899}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 0.4401596486568451, "loss_scale": 4.0, "retries": 0, "seconds": 143.4, "max_memory_gb": 17.17, "ce": 0.10218888576491736, "kl": 0.21966026289351248, "context": 0.21201902470784262}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.2302914559841156, "loss_scale": 4.0, "retries": 0, "seconds": 144.7, "max_memory_gb": 17.17, "ce": 0.011555089615285397, "kl": 0.008391468083573272, "context": 0.015750823673442937}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 0.9010377526283264, "loss_scale": 4.0, "retries": 0, "seconds": 146.0, "max_memory_gb": 17.17, "ce": 0.2830549227510346, "kl": 0.004487590671487851, "context": 0.2852987264341209}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.2042669802904129, "loss_scale": 4.0, "retries": 0, "seconds": 147.1, "max_memory_gb": 17.17, "ce": 0.01857961783389328, "kl": 0.016399422328049695, "context": 0.02677932899678126}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 2.244381904602051, "loss_scale": 4.0, "retries": 0, "seconds": 148.5, "max_memory_gb": 17.17, "ce": 0.15028868155786768, "kl": 0.16848074694280513, "context": 0.2345290568191558}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.495490163564682, "loss_scale": 4.0, "retries": 0, "seconds": 149.6, "max_memory_gb": 17.17, "ce": 0.09687411398772383, "kl": 0.04294337371902657, "context": 0.11834580084541813}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.0132253170013428, "loss_scale": 4.0, "retries": 0, "seconds": 150.8, "max_memory_gb": 17.17, "ce": 0.16079488267132547, "kl": 0.02149009651839151, "context": 0.17153993509418797}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 1.2140508890151978, "loss_scale": 4.0, "retries": 0, "seconds": 152.1, "max_memory_gb": 17.17, "ce": 0.058733595069497824, "kl": 0.046174824063200504, "context": 0.08182100532576442}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 1.7246472835540771, "loss_scale": 4.0, "retries": 0, "seconds": 153.5, "max_memory_gb": 17.17, "ce": 0.6194883030548226, "kl": 0.42342341434323316, "context": 0.8311999925062992}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.4449476897716522, "loss_scale": 4.0, "retries": 0, "seconds": 154.8, "max_memory_gb": 17.17, "ce": 0.1504652264120523, "kl": 0.053553415287751704, "context": 0.17724192677997053}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 1.8665691614151, "loss_scale": 4.0, "retries": 0, "seconds": 156.0, "max_memory_gb": 17.17, "ce": 0.6307596561964601, "kl": 0.16718341386877, "context": 0.7143513574264944}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.02024846337735653, "loss_scale": 4.0, "retries": 0, "seconds": 157.2, "max_memory_gb": 17.17, "ce": 0.0014324385992949829, "kl": 0.0002771078325167764, "context": 0.001570992506458424}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 1.4315677881240845, "loss_scale": 4.0, "retries": 0, "seconds": 158.6, "max_memory_gb": 17.17, "ce": 0.12543094166903757, "kl": 0.44627376439166255, "context": 0.34856782405404374}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 1.435054898262024, "loss_scale": 4.0, "retries": 0, "seconds": 159.8, "max_memory_gb": 17.17, "ce": 0.09085330905509181, "kl": 0.046731498192457366, "context": 0.11421905997849535}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.6418062448501587, "loss_scale": 4.0, "retries": 0, "seconds": 160.9, "max_memory_gb": 17.17, "ce": 0.03458515554666519, "kl": 0.34030825992886093, "context": 0.2047392794629559}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 1.7723246812820435, "loss_scale": 4.0, "retries": 0, "seconds": 162.2, "max_memory_gb": 17.17, "ce": 0.23855444003129378, "kl": 0.2576670278212987, "context": 0.3673879705602303}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 1.8856477737426758, "loss_scale": 4.0, "retries": 0, "seconds": 163.4, "max_memory_gb": 17.17, "ce": 0.21060430513171013, "kl": 0.033165693374030525, "context": 0.22718714717484545}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.9224534630775452, "loss_scale": 4.0, "retries": 0, "seconds": 164.5, "max_memory_gb": 17.17, "ce": 0.27485009652082226, "kl": 0.008661898202262819, "context": 0.27918104105629027}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.6254575252532959, "loss_scale": 4.0, "retries": 0, "seconds": 165.9, "max_memory_gb": 17.17, "ce": 0.1653976356246858, "kl": 0.0019456103266293212, "context": 0.16637044603703544}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 0.3742566704750061, "loss_scale": 4.0, "retries": 0, "seconds": 167.0, "max_memory_gb": 17.17, "ce": 0.010644078342011198, "kl": 0.029793241406878224, "context": 0.025540699512930587}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 0.848394513130188, "loss_scale": 4.0, "retries": 0, "seconds": 168.3, "max_memory_gb": 17.17, "ce": 0.051726081350352615, "kl": 0.059865131122933235, "context": 0.08165864303009585}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 0.3848358392715454, "loss_scale": 4.0, "retries": 0, "seconds": 169.5, "max_memory_gb": 17.17, "ce": 0.03034283201850485, "kl": 0.03925115794118028, "context": 0.049968409090070054}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 2.7681028842926025, "loss_scale": 4.0, "retries": 0, "seconds": 170.7, "max_memory_gb": 17.17, "ce": 0.11204583221115172, "kl": 0.20106892008334398, "context": 0.2125802922528237}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 0.46688568592071533, "loss_scale": 4.0, "retries": 0, "seconds": 172.0, "max_memory_gb": 17.17, "ce": 0.01020808813336771, "kl": 0.11564479436492547, "context": 0.068030488997465}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.07253945618867874, "loss_scale": 4.0, "retries": 0, "seconds": 173.2, "max_memory_gb": 17.17, "ce": 0.006950749637326226, "kl": 0.002780344177153893, "context": 0.008340921514900401}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.06820055097341537, "loss_scale": 4.0, "retries": 0, "seconds": 174.3, "max_memory_gb": 17.17, "ce": 0.008462791880447185, "kl": 0.0006754665037078667, "context": 0.008800525185506558}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 0.3246577978134155, "loss_scale": 4.0, "retries": 0, "seconds": 175.6, "max_memory_gb": 17.17, "ce": 0.029550600855145603, "kl": 0.012018402892863378, "context": 0.035559802141506225}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 1.5252171754837036, "loss_scale": 4.0, "retries": 0, "seconds": 176.8, "max_memory_gb": 17.17, "ce": 0.29067018906062003, "kl": 0.038070105665610754, "context": 0.3097052424127469}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.6809080839157104, "loss_scale": 4.0, "retries": 0, "seconds": 178.0, "max_memory_gb": 17.17, "ce": 0.04679104400565848, "kl": 0.11387254227884114, "context": 0.10372731456300244}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.06230277940630913, "loss_scale": 4.0, "retries": 0, "seconds": 179.3, "max_memory_gb": 17.17, "ce": 0.0015314090705942363, "kl": 0.005744065045291791, "context": 0.004403441591421142}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.3150610625743866, "loss_scale": 4.0, "retries": 0, "seconds": 180.4, "max_memory_gb": 17.17, "ce": 0.028018508397508413, "kl": 0.015567417945931084, "context": 0.035802216618321836}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 2.0035037994384766, "loss_scale": 4.0, "retries": 0, "seconds": 181.7, "max_memory_gb": 17.17, "ce": 0.3650016735191457, "kl": 0.3772731379576726, "context": 0.5536382386344485}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.1621292680501938, "loss_scale": 4.0, "retries": 0, "seconds": 182.9, "max_memory_gb": 17.17, "ce": 0.00934554597188253, "kl": 0.013975089357700199, "context": 0.016333090970874764}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 0.19239163398742676, "loss_scale": 4.0, "retries": 0, "seconds": 184.1, "max_memory_gb": 17.17, "ce": 0.02393083501374349, "kl": 0.0015655986808269517, "context": 0.024713634193176404}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.35061389207839966, "loss_scale": 4.0, "retries": 0, "seconds": 185.4, "max_memory_gb": 17.17, "ce": 0.06308448548952583, "kl": 0.01349357855906419, "context": 0.06983127756393515}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.5456501841545105, "loss_scale": 4.0, "retries": 0, "seconds": 186.6, "max_memory_gb": 17.17, "ce": 0.012625848030438647, "kl": 0.018400740480501554, "context": 0.021826219162903726}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 1.5666942596435547, "loss_scale": 4.0, "retries": 0, "seconds": 187.7, "max_memory_gb": 17.17, "ce": 0.005713934529921971, "kl": 0.5335091592482968, "context": 0.27246850164374337}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.8546605706214905, "loss_scale": 4.0, "retries": 0, "seconds": 189.0, "max_memory_gb": 17.17, "ce": 0.0072012469681794755, "kl": 0.5473670548799419, "context": 0.280884783147485}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.6877098083496094, "loss_scale": 4.0, "retries": 0, "seconds": 190.1, "max_memory_gb": 17.17, "ce": 0.0339106808823999, "kl": 0.07278423327079508, "context": 0.07030279567698017}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 2.341421365737915, "loss_scale": 4.0, "retries": 0, "seconds": 191.3, "max_memory_gb": 17.17, "ce": 0.29377179472794523, "kl": 0.01459119159335387, "context": 0.3010674036195269}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.7215952277183533, "loss_scale": 4.0, "retries": 0, "seconds": 192.7, "max_memory_gb": 17.17, "ce": 0.03881182059558341, "kl": 0.006231372189176909, "context": 0.04192750505899312}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.48571234941482544, "loss_scale": 4.0, "retries": 0, "seconds": 193.8, "max_memory_gb": 17.17, "ce": 0.030586532084271312, "kl": 0.04581892399437493, "context": 0.05349599476903677}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.057472825050354, "loss_scale": 4.0, "retries": 0, "seconds": 195.1, "max_memory_gb": 17.17, "ce": 0.23382426978787407, "kl": 0.07229835007456131, "context": 0.2699734370689839}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 1.4283993244171143, "loss_scale": 4.0, "retries": 0, "seconds": 196.1, "max_memory_gb": 17.17, "ce": 0.04544021066976711, "kl": 0.09817048635886749, "context": 0.09452545479871333}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 1.8363748788833618, "loss_scale": 4.0, "retries": 0, "seconds": 197.1, "max_memory_gb": 17.17, "ce": 0.08177297990187071, "kl": 0.4675472019298468, "context": 0.31554658274399117}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 1.6411335468292236, "loss_scale": 4.0, "retries": 0, "seconds": 198.2, "max_memory_gb": 17.17, "ce": 0.1508958459016867, "kl": 0.12019956241056207, "context": 0.21099562988092657}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 1.1930755376815796, "loss_scale": 4.0, "retries": 0, "seconds": 199.4, "max_memory_gb": 17.17, "ce": 0.1153716549888486, "kl": 0.16990801657084376, "context": 0.20032566323061474}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 3.611849546432495, "loss_scale": 4.0, "retries": 0, "seconds": 200.5, "max_memory_gb": 17.17, "ce": 0.2864777326103649, "kl": 0.69442832621462, "context": 0.6336918968809186}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.8291441798210144, "loss_scale": 4.0, "retries": 0, "seconds": 201.9, "max_memory_gb": 17.17, "ce": 0.1228084122412838, "kl": 0.06911329376816866, "context": 0.15736506191751687}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.17929305136203766, "loss_scale": 4.0, "retries": 0, "seconds": 203.1, "max_memory_gb": 17.17, "ce": 0.013538177619921044, "kl": 0.008493337856634753, "context": 0.017784846655558795}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 1.621791958808899, "loss_scale": 4.0, "retries": 0, "seconds": 204.2, "max_memory_gb": 17.17, "ce": 0.35523274436127394, "kl": 0.07112796074943617, "context": 0.3907967173727229}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.8963177800178528, "loss_scale": 4.0, "retries": 0, "seconds": 205.6, "max_memory_gb": 17.17, "ce": 0.036897899757605046, "kl": 0.01725137696485035, "context": 0.045523586217314005}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.0741063728928566, "loss_scale": 4.0, "retries": 0, "seconds": 206.8, "max_memory_gb": 17.17, "ce": 0.007087765807227697, "kl": 0.0013944303573225625, "context": 0.0077849807785241865}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 1.448339819908142, "loss_scale": 4.0, "retries": 0, "seconds": 208.1, "max_memory_gb": 17.17, "ce": 0.07963591019506566, "kl": 0.9894316755235195, "context": 0.5743517405353487}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.011044066399335861, "loss_scale": 4.0, "retries": 0, "seconds": 209.3, "max_memory_gb": 17.17, "ce": 0.0009484397014603019, "kl": 0.00015682032517361222, "context": 0.0010268498590448871}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.5916393399238586, "loss_scale": 4.0, "retries": 0, "seconds": 210.4, "max_memory_gb": 17.17, "ce": 0.018315283090487355, "kl": 0.0547605799511075, "context": 0.0456955733243376}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.23444953560829163, "loss_scale": 4.0, "retries": 0, "seconds": 211.5, "max_memory_gb": 17.17, "ce": 0.004880985943600535, "kl": 0.021372100105509162, "context": 0.015567035996355116}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.29242607951164246, "loss_scale": 4.0, "retries": 0, "seconds": 212.6, "max_memory_gb": 17.17, "ce": 0.01645482037565671, "kl": 0.0034771543359966017, "context": 0.018193397554568946}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.8046543002128601, "loss_scale": 4.0, "retries": 0, "seconds": 213.8, "max_memory_gb": 17.17, "ce": 0.04572465185628971, "kl": 0.017295567311407467, "context": 0.054372434664401226}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 1.2328717708587646, "loss_scale": 4.0, "retries": 0, "seconds": 215.0, "max_memory_gb": 17.17, "ce": 0.16404905798844993, "kl": 0.03595857380423695, "context": 0.18202834483236074}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.04849863797426224, "loss_scale": 4.0, "retries": 0, "seconds": 216.2, "max_memory_gb": 17.17, "ce": 0.0033375245693605393, "kl": 0.000957695156103, "context": 0.003816372074652463}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.7655101418495178, "loss_scale": 4.0, "retries": 0, "seconds": 217.4, "max_memory_gb": 17.17, "ce": 0.041943148273276165, "kl": 0.7535008476370422, "context": 0.418693561834516}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.3443300724029541, "loss_scale": 4.0, "retries": 0, "seconds": 218.5, "max_memory_gb": 17.17, "ce": 0.08126789328525774, "kl": 0.03239665599539876, "context": 0.09746622084639966}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 0.7303569912910461, "loss_scale": 4.0, "retries": 0, "seconds": 219.7, "max_memory_gb": 17.17, "ce": 0.3834742694743909, "kl": 0.009832888270466356, "context": 0.3883907117706258}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.40673330426216125, "loss_scale": 4.0, "retries": 0, "seconds": 221.0, "max_memory_gb": 17.17, "ce": 0.027674103461322375, "kl": 0.01758121303282678, "context": 0.0364647091482766}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 0.8850833177566528, "loss_scale": 4.0, "retries": 0, "seconds": 222.1, "max_memory_gb": 17.17, "ce": 0.0037063498530187644, "kl": 0.17063088667055126, "context": 0.0890217955748085}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.7791585922241211, "loss_scale": 4.0, "retries": 0, "seconds": 223.3, "max_memory_gb": 17.17, "ce": 0.06730972579680383, "kl": 0.2117179761062289, "context": 0.17316870918148197}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.02054680325090885, "loss_scale": 4.0, "retries": 0, "seconds": 224.6, "max_memory_gb": 17.17, "ce": 0.001788605484762229, "kl": 0.0002723038378462661, "context": 0.0019247574236942455}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.046755459159612656, "loss_scale": 4.0, "retries": 0, "seconds": 225.8, "max_memory_gb": 17.17, "ce": 0.0025932076532626525, "kl": 0.0027666354035318363, "context": 0.0039765252149663866}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.7348775863647461, "loss_scale": 4.0, "retries": 0, "seconds": 226.9, "max_memory_gb": 17.17, "ce": 0.005821058701258153, "kl": 0.15886524895904586, "context": 0.08525367960100994}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 1.3337676525115967, "loss_scale": 4.0, "retries": 0, "seconds": 228.1, "max_memory_gb": 17.17, "ce": 0.11550780403922545, "kl": 0.203167304280214, "context": 0.21709146400826285}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 0.8345776200294495, "loss_scale": 4.0, "retries": 0, "seconds": 229.1, "max_memory_gb": 17.17, "ce": 0.0168796896468848, "kl": 0.5907320742844604, "context": 0.3122457338613458}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 0.8081107139587402, "loss_scale": 4.0, "retries": 0, "seconds": 230.3, "max_memory_gb": 17.17, "ce": 0.23831413600419182, "kl": 0.01694367236632388, "context": 0.2467859651951585}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 1.1436333656311035, "loss_scale": 4.0, "retries": 0, "seconds": 231.6, "max_memory_gb": 17.17, "ce": 0.32222982932580635, "kl": 0.2307173918306944, "context": 0.43758853641338646}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.021973051130771637, "loss_scale": 4.0, "retries": 0, "seconds": 232.7, "max_memory_gb": 17.17, "ce": 0.0011606832267716527, "kl": 0.0007700861388002522, "context": 0.0015457262779818848}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 1.2787071466445923, "loss_scale": 4.0, "retries": 0, "seconds": 233.8, "max_memory_gb": 17.17, "ce": 0.26205070258583874, "kl": 0.12882527078363637, "context": 0.32646333042066544}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.0984406471252441, "loss_scale": 4.0, "retries": 0, "seconds": 234.9, "max_memory_gb": 17.17, "ce": 0.42466153742861934, "kl": 0.0810182077857462, "context": 0.4651706548174843}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.036923617124557495, "loss_scale": 4.0, "retries": 0, "seconds": 236.0, "max_memory_gb": 17.17, "ce": 0.002666613072506152, "kl": 0.001559746258863015, "context": 0.003446486225584522}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.4216868281364441, "loss_scale": 4.0, "retries": 0, "seconds": 237.2, "max_memory_gb": 17.17, "ce": 0.07359765440924093, "kl": 0.05387709207843727, "context": 0.10053619695827365}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.10618511587381363, "loss_scale": 4.0, "retries": 0, "seconds": 238.3, "max_memory_gb": 17.17, "ce": 0.013985197787405923, "kl": 0.0021196947564021684, "context": 0.015045045205624774}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 1.3614286184310913, "loss_scale": 4.0, "retries": 0, "seconds": 239.4, "max_memory_gb": 17.17, "ce": 0.0015564733985229395, "kl": 0.19651632138413788, "context": 0.09981463351141429}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 1.3605514764785767, "loss_scale": 4.0, "retries": 0, "seconds": 240.6, "max_memory_gb": 17.17, "ce": 0.07181623319047503, "kl": 0.6615924726247613, "context": 0.4026124707306735}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.41695526242256165, "loss_scale": 4.0, "retries": 0, "seconds": 241.7, "max_memory_gb": 17.17, "ce": 0.09492115968896542, "kl": 0.006749767824658193, "context": 0.0982960434484994}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.6752862930297852, "loss_scale": 4.0, "retries": 0, "seconds": 242.9, "max_memory_gb": 17.17, "ce": 0.10872498295793775, "kl": 0.004330197483795928, "context": 0.11089008179260418}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.6320797204971313, "loss_scale": 4.0, "retries": 0, "seconds": 244.2, "max_memory_gb": 17.17, "ce": 0.2582071844371967, "kl": 0.02681066144577926, "context": 0.27161251683719456}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.5678330063819885, "loss_scale": 4.0, "retries": 0, "seconds": 245.4, "max_memory_gb": 17.17, "ce": 0.07711806186125614, "kl": 0.1043078816801426, "context": 0.12927200258127414}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.0328667126595974, "loss_scale": 4.0, "retries": 0, "seconds": 246.7, "max_memory_gb": 17.17, "ce": 0.0018380228502792306, "kl": 0.0023165852740021364, "context": 0.002996315488417167}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.27737152576446533, "loss_scale": 4.0, "retries": 0, "seconds": 247.9, "max_memory_gb": 17.17, "ce": 0.01925751392263919, "kl": 0.018608041194966063, "context": 0.0285615345928818}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.1592673361301422, "loss_scale": 4.0, "retries": 0, "seconds": 249.0, "max_memory_gb": 17.17, "ce": 0.01117481803521514, "kl": 0.00461875883047469, "context": 0.013484197144862264}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 1.1138784885406494, "loss_scale": 4.0, "retries": 0, "seconds": 250.3, "max_memory_gb": 17.17, "ce": 0.004879677828284912, "kl": 0.39515761913207825, "context": 0.2024584861646872}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.03777071088552475, "loss_scale": 4.0, "retries": 0, "seconds": 251.3, "max_memory_gb": 17.17, "ce": 0.002235359548649285, "kl": 0.0012578098649100866, "context": 0.002864264417439699}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.34842461347579956, "loss_scale": 4.0, "retries": 0, "seconds": 252.3, "max_memory_gb": 17.17, "ce": 0.08483929396606982, "kl": 0.043690498161595315, "context": 0.10668454656843096}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.07164427638053894, "loss_scale": 4.0, "retries": 0, "seconds": 253.4, "max_memory_gb": 17.17, "ce": 0.004501330782659352, "kl": 0.0015061304966366151, "context": 0.005254396062809974}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.701663613319397, "loss_scale": 4.0, "retries": 0, "seconds": 254.5, "max_memory_gb": 17.17, "ce": 0.17933276438270696, "kl": 0.03826221560302656, "context": 0.19846387204597704}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 0.987954318523407, "loss_scale": 4.0, "retries": 0, "seconds": 255.7, "max_memory_gb": 17.17, "ce": 0.47495988329319516, "kl": 0.0962879936923855, "context": 0.5231038949859794}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.31506475806236267, "loss_scale": 4.0, "retries": 0, "seconds": 257.0, "max_memory_gb": 17.17, "ce": 0.10714741318952292, "kl": 0.08965169939256157, "context": 0.1519732558663236}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 1.4163920879364014, "loss_scale": 4.0, "retries": 0, "seconds": 258.2, "max_memory_gb": 17.17, "ce": 0.08312123283394612, "kl": 0.8812529965798603, "context": 0.5237477171467617}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.610927164554596, "loss_scale": 4.0, "retries": 0, "seconds": 259.5, "max_memory_gb": 17.17, "ce": 0.20090961814275943, "kl": 0.023448548545275116, "context": 0.2126338998787105}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 2.525634527206421, "loss_scale": 4.0, "retries": 0, "seconds": 260.7, "max_memory_gb": 17.17, "ce": 0.3431636616878677, "kl": 0.08756635640747845, "context": 0.3869468446355313}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 1.1381407976150513, "loss_scale": 4.0, "retries": 0, "seconds": 261.8, "max_memory_gb": 17.17, "ce": 0.4877461838477757, "kl": 0.024749544758492448, "context": 0.5001209645706695}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 1.0476963520050049, "loss_scale": 4.0, "retries": 0, "seconds": 263.1, "max_memory_gb": 17.17, "ce": 0.02704529243055731, "kl": 0.17677212536364095, "context": 0.11543135368265212}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.03009088709950447, "loss_scale": 4.0, "retries": 0, "seconds": 264.3, "max_memory_gb": 17.17, "ce": 0.0027707960980478674, "kl": 0.001251096951477848, "context": 0.003396344560314901}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 0.6098612546920776, "loss_scale": 4.0, "retries": 0, "seconds": 265.5, "max_memory_gb": 17.17, "ce": 0.17262533027678728, "kl": 0.1484958678483963, "context": 0.246873265132308}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.2628982663154602, "loss_scale": 4.0, "retries": 0, "seconds": 266.8, "max_memory_gb": 17.17, "ce": 0.016200560377910733, "kl": 0.09933605304104276, "context": 0.06586858618538827}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 0.1447935551404953, "loss_scale": 4.0, "retries": 0, "seconds": 267.9, "max_memory_gb": 17.17, "ce": 0.011552053212653846, "kl": 0.002546448390603473, "context": 0.012825276935473084}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.8940602540969849, "loss_scale": 4.0, "retries": 0, "seconds": 268.9, "max_memory_gb": 17.17, "ce": 0.0027994435222353786, "kl": 0.09447128744250222, "context": 0.05003508829395287}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.9111977815628052, "loss_scale": 4.0, "retries": 0, "seconds": 270.0, "max_memory_gb": 17.17, "ce": 0.03416727975127287, "kl": 0.03670242847010741, "context": 0.052518494543619454}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 1.1838207244873047, "loss_scale": 4.0, "retries": 0, "seconds": 271.6, "max_memory_gb": 17.17, "ce": 0.13838681284687482, "kl": 0.009660452235038974, "context": 0.1432170347252395}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.7213457226753235, "loss_scale": 4.0, "retries": 0, "seconds": 273.0, "max_memory_gb": 17.17, "ce": 0.06089513027109206, "kl": 0.06630754847356002, "context": 0.09404890798032284}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.3274647295475006, "loss_scale": 4.0, "retries": 0, "seconds": 274.2, "max_memory_gb": 17.17, "ce": 0.006845470023108646, "kl": 0.027718577002815437, "context": 0.02070475887740031}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.5293148159980774, "loss_scale": 4.0, "retries": 0, "seconds": 275.4, "max_memory_gb": 17.17, "ce": 0.04853493164409883, "kl": 0.013125889738148544, "context": 0.05509787835762836}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.28395920991897583, "loss_scale": 4.0, "retries": 0, "seconds": 276.7, "max_memory_gb": 17.17, "ce": 0.0317718611768214, "kl": 0.010692757157812594, "context": 0.037118238338734955}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.30336666107177734, "loss_scale": 4.0, "retries": 0, "seconds": 277.9, "max_memory_gb": 17.17, "ce": 0.013465223644743674, "kl": 0.028199283173307776, "context": 0.027564865071326494}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 0.5743948817253113, "loss_scale": 4.0, "retries": 0, "seconds": 279.2, "max_memory_gb": 17.17, "ce": 0.08919886223156936, "kl": 0.027885683919976145, "context": 0.10314170466153882}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 1.5106315612792969, "loss_scale": 4.0, "retries": 0, "seconds": 280.4, "max_memory_gb": 17.17, "ce": 0.4172187414369546, "kl": 0.23507299809716642, "context": 0.5347552332095802}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.5437212586402893, "loss_scale": 4.0, "retries": 0, "seconds": 281.4, "max_memory_gb": 17.17, "ce": 0.04768705632886849, "kl": 0.07344143361842725, "context": 0.08440777464420535}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.4932134747505188, "loss_scale": 4.0, "retries": 0, "seconds": 282.5, "max_memory_gb": 17.17, "ce": 0.02531382002052851, "kl": 0.011985641496721655, "context": 0.031306640827097}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.8576617240905762, "loss_scale": 4.0, "retries": 0, "seconds": 283.7, "max_memory_gb": 17.17, "ce": 0.016856422764249146, "kl": 0.15726120950421318, "context": 0.09548702707979828}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.22822783887386322, "loss_scale": 4.0, "retries": 0, "seconds": 284.9, "max_memory_gb": 17.17, "ce": 0.016124081215821207, "kl": 0.022204827175301034, "context": 0.027226494741626084}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 1.1906509399414062, "loss_scale": 4.0, "retries": 0, "seconds": 286.1, "max_memory_gb": 17.17, "ce": 0.2967374254949391, "kl": 0.020910483166517224, "context": 0.3071926624106709}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.13400773704051971, "loss_scale": 4.0, "retries": 0, "seconds": 287.1, "max_memory_gb": 17.17, "ce": 0.011999558832030743, "kl": 0.010180579367442988, "context": 0.01708984829019755}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.4404430389404297, "loss_scale": 4.0, "retries": 0, "seconds": 288.3, "max_memory_gb": 17.17, "ce": 0.09045262302970514, "kl": 0.023132905933607617, "context": 0.10201907809823751}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.22602027654647827, "loss_scale": 4.0, "retries": 0, "seconds": 289.5, "max_memory_gb": 17.17, "ce": 0.0258348767310963, "kl": 0.006877036675177806, "context": 0.029273396023199894}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.38312995433807373, "loss_scale": 4.0, "retries": 0, "seconds": 290.6, "max_memory_gb": 17.17, "ce": 0.10611355619039387, "kl": 0.013434938300633803, "context": 0.11283102806191891}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 1.004941701889038, "loss_scale": 4.0, "retries": 0, "seconds": 291.7, "max_memory_gb": 17.17, "ce": 0.07155561505351216, "kl": 0.027928116673137993, "context": 0.08551967534003779}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.6549524068832397, "loss_scale": 4.0, "retries": 0, "seconds": 293.0, "max_memory_gb": 17.17, "ce": 0.18756708248110954, "kl": 0.021150303073227406, "context": 0.19814223151479382}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.13964082300662994, "loss_scale": 4.0, "retries": 0, "seconds": 294.1, "max_memory_gb": 17.17, "ce": 0.011596616823226213, "kl": 0.012089955697547339, "context": 0.01764159437152557}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.027527255937457085, "loss_scale": 4.0, "retries": 0, "seconds": 295.2, "max_memory_gb": 17.17, "ce": 0.002241319540189579, "kl": 0.0013264674271340482, "context": 0.002904553199186921}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 0.6525247097015381, "loss_scale": 4.0, "retries": 0, "seconds": 296.7, "max_memory_gb": 17.17, "ce": 0.04821555160742719, "kl": 0.12105363168211625, "context": 0.10874236628296785}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.7651572227478027, "loss_scale": 4.0, "retries": 0, "seconds": 297.9, "max_memory_gb": 17.17, "ce": 0.3628020916803507, "kl": 0.04305742982251104, "context": 0.3843308141513262}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.0331629179418087, "loss_scale": 4.0, "retries": 0, "seconds": 299.2, "max_memory_gb": 17.17, "ce": 0.002583730311016552, "kl": 0.0006625045025430154, "context": 0.002914982571383007}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 1.315447449684143, "loss_scale": 4.0, "retries": 0, "seconds": 300.4, "max_memory_gb": 17.17, "ce": 0.010847874393220991, "kl": 0.32863646338228136, "context": 0.17516611239989288}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.26374366879463196, "loss_scale": 4.0, "retries": 0, "seconds": 301.6, "max_memory_gb": 17.17, "ce": 0.018962105794344097, "kl": 0.016673276550136507, "context": 0.027298744302242994}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.6211029887199402, "loss_scale": 4.0, "retries": 0, "seconds": 303.0, "max_memory_gb": 17.17, "ce": 0.03100784868001938, "kl": 0.028340452525299042, "context": 0.04517807625234127}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.09143319725990295, "loss_scale": 4.0, "retries": 0, "seconds": 304.1, "max_memory_gb": 17.17, "ce": 0.009566727734636515, "kl": 0.004660507052903995, "context": 0.011896981159225106}

runs/fixed_internvl/context/Deep8_s42/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Deep8_s42/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.28.mlp.down_proj": 144,
+    "model.language_model.layers.28.mlp.gate_proj": 144,
+    "model.language_model.layers.28.mlp.up_proj": 144,
+    "model.language_model.layers.28.self_attn.k_proj": 144,
+    "model.language_model.layers.28.self_attn.o_proj": 144,
+    "model.language_model.layers.28.self_attn.q_proj": 144,
+    "model.language_model.layers.28.self_attn.v_proj": 144,
+    "model.language_model.layers.29.mlp.down_proj": 144,
+    "model.language_model.layers.29.mlp.gate_proj": 144,
+    "model.language_model.layers.29.mlp.up_proj": 144,
+    "model.language_model.layers.29.self_attn.k_proj": 144,
+    "model.language_model.layers.29.self_attn.o_proj": 144,
+    "model.language_model.layers.29.self_attn.q_proj": 144,
+    "model.language_model.layers.29.self_attn.v_proj": 144,
+    "model.language_model.layers.30.mlp.down_proj": 144,
+    "model.language_model.layers.30.mlp.gate_proj": 144,
+    "model.language_model.layers.30.mlp.up_proj": 144,
+    "model.language_model.layers.30.self_attn.k_proj": 144,
+    "model.language_model.layers.30.self_attn.o_proj": 144,
+    "model.language_model.layers.30.self_attn.q_proj": 144,
+    "model.language_model.layers.30.self_attn.v_proj": 144,
+    "model.language_model.layers.31.mlp.down_proj": 144,
+    "model.language_model.layers.31.mlp.gate_proj": 144,
+    "model.language_model.layers.31.mlp.up_proj": 144,
+    "model.language_model.layers.31.self_attn.k_proj": 144,
+    "model.language_model.layers.31.self_attn.o_proj": 144,
+    "model.language_model.layers.31.self_attn.q_proj": 144,
+    "model.language_model.layers.31.self_attn.v_proj": 144,
+    "model.language_model.layers.32.mlp.down_proj": 144,
+    "model.language_model.layers.32.mlp.gate_proj": 144,
+    "model.language_model.layers.32.mlp.up_proj": 144,
+    "model.language_model.layers.32.self_attn.k_proj": 144,
+    "model.language_model.layers.32.self_attn.o_proj": 144,
+    "model.language_model.layers.32.self_attn.q_proj": 144,
+    "model.language_model.layers.32.self_attn.v_proj": 144,
+    "model.language_model.layers.33.mlp.down_proj": 144,
+    "model.language_model.layers.33.mlp.gate_proj": 144,
+    "model.language_model.layers.33.mlp.up_proj": 144,
+    "model.language_model.layers.33.self_attn.k_proj": 144,
+    "model.language_model.layers.33.self_attn.o_proj": 144,
+    "model.language_model.layers.33.self_attn.q_proj": 144,
+    "model.language_model.layers.33.self_attn.v_proj": 144,
+    "model.language_model.layers.34.mlp.down_proj": 144,
+    "model.language_model.layers.34.mlp.gate_proj": 144,
+    "model.language_model.layers.34.mlp.up_proj": 144,
+    "model.language_model.layers.34.self_attn.k_proj": 144,
+    "model.language_model.layers.34.self_attn.o_proj": 144,
+    "model.language_model.layers.34.self_attn.q_proj": 144,
+    "model.language_model.layers.34.self_attn.v_proj": 144,
+    "model.language_model.layers.35.mlp.down_proj": 144,
+    "model.language_model.layers.35.mlp.gate_proj": 144,
+    "model.language_model.layers.35.mlp.up_proj": 144,
+    "model.language_model.layers.35.self_attn.k_proj": 144,
+    "model.language_model.layers.35.self_attn.o_proj": 144,
+    "model.language_model.layers.35.self_attn.q_proj": 144,
+    "model.language_model.layers.35.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.28.mlp.down_proj": 72,
+    "model.language_model.layers.28.mlp.gate_proj": 72,
+    "model.language_model.layers.28.mlp.up_proj": 72,
+    "model.language_model.layers.28.self_attn.k_proj": 72,
+    "model.language_model.layers.28.self_attn.o_proj": 72,
+    "model.language_model.layers.28.self_attn.q_proj": 72,
+    "model.language_model.layers.28.self_attn.v_proj": 72,
+    "model.language_model.layers.29.mlp.down_proj": 72,
+    "model.language_model.layers.29.mlp.gate_proj": 72,
+    "model.language_model.layers.29.mlp.up_proj": 72,
+    "model.language_model.layers.29.self_attn.k_proj": 72,
+    "model.language_model.layers.29.self_attn.o_proj": 72,
+    "model.language_model.layers.29.self_attn.q_proj": 72,
+    "model.language_model.layers.29.self_attn.v_proj": 72,
+    "model.language_model.layers.30.mlp.down_proj": 72,
+    "model.language_model.layers.30.mlp.gate_proj": 72,
+    "model.language_model.layers.30.mlp.up_proj": 72,
+    "model.language_model.layers.30.self_attn.k_proj": 72,
+    "model.language_model.layers.30.self_attn.o_proj": 72,
+    "model.language_model.layers.30.self_attn.q_proj": 72,
+    "model.language_model.layers.30.self_attn.v_proj": 72,
+    "model.language_model.layers.31.mlp.down_proj": 72,
+    "model.language_model.layers.31.mlp.gate_proj": 72,
+    "model.language_model.layers.31.mlp.up_proj": 72,
+    "model.language_model.layers.31.self_attn.k_proj": 72,
+    "model.language_model.layers.31.self_attn.o_proj": 72,
+    "model.language_model.layers.31.self_attn.q_proj": 72,
+    "model.language_model.layers.31.self_attn.v_proj": 72,
+    "model.language_model.layers.32.mlp.down_proj": 72,
+    "model.language_model.layers.32.mlp.gate_proj": 72,
+    "model.language_model.layers.32.mlp.up_proj": 72,
+    "model.language_model.layers.32.self_attn.k_proj": 72,
+    "model.language_model.layers.32.self_attn.o_proj": 72,
+    "model.language_model.layers.32.self_attn.q_proj": 72,
+    "model.language_model.layers.32.self_attn.v_proj": 72,
+    "model.language_model.layers.33.mlp.down_proj": 72,
+    "model.language_model.layers.33.mlp.gate_proj": 72,
+    "model.language_model.layers.33.mlp.up_proj": 72,
+    "model.language_model.layers.33.self_attn.k_proj": 72,
+    "model.language_model.layers.33.self_attn.o_proj": 72,
+    "model.language_model.layers.33.self_attn.q_proj": 72,
+    "model.language_model.layers.33.self_attn.v_proj": 72,
+    "model.language_model.layers.34.mlp.down_proj": 72,
+    "model.language_model.layers.34.mlp.gate_proj": 72,
+    "model.language_model.layers.34.mlp.up_proj": 72,
+    "model.language_model.layers.34.self_attn.k_proj": 72,
+    "model.language_model.layers.34.self_attn.o_proj": 72,
+    "model.language_model.layers.34.self_attn.q_proj": 72,
+    "model.language_model.layers.34.self_attn.v_proj": 72,
+    "model.language_model.layers.35.mlp.down_proj": 72,
+    "model.language_model.layers.35.mlp.gate_proj": 72,
+    "model.language_model.layers.35.mlp.up_proj": 72,
+    "model.language_model.layers.35.self_attn.k_proj": 72,
+    "model.language_model.layers.35.self_attn.o_proj": 72,
+    "model.language_model.layers.35.self_attn.q_proj": 72,
+    "model.language_model.layers.35.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.34.mlp.down_proj",
+    "model.language_model.layers.34.mlp.up_proj",
+    "model.language_model.layers.33.self_attn.q_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.31.mlp.up_proj",
+    "model.language_model.layers.32.self_attn.k_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.33.mlp.gate_proj",
+    "model.language_model.layers.35.self_attn.k_proj",
+    "model.language_model.layers.32.self_attn.o_proj",
+    "model.language_model.layers.29.self_attn.v_proj",
+    "model.language_model.layers.35.mlp.down_proj",
+    "model.language_model.layers.32.mlp.up_proj",
+    "model.language_model.layers.30.mlp.gate_proj",
+    "model.language_model.layers.32.self_attn.q_proj",
+    "model.language_model.layers.31.mlp.down_proj",
+    "model.language_model.layers.30.self_attn.k_proj",
+    "model.language_model.layers.31.self_attn.q_proj",
+    "model.language_model.layers.32.mlp.gate_proj",
+    "model.language_model.layers.35.self_attn.v_proj",
+    "model.language_model.layers.30.mlp.down_proj",
+    "model.language_model.layers.35.mlp.gate_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.30.self_attn.q_proj",
+    "model.language_model.layers.33.mlp.up_proj",
+    "model.language_model.layers.34.self_attn.k_proj",
+    "model.language_model.layers.35.self_attn.q_proj",
+    "model.language_model.layers.31.self_attn.k_proj",
+    "model.language_model.layers.34.self_attn.q_proj",
+    "model.language_model.layers.29.self_attn.k_proj",
+    "model.language_model.layers.33.mlp.down_proj",
+    "model.language_model.layers.28.mlp.gate_proj",
+    "model.language_model.layers.31.mlp.gate_proj",
+    "model.language_model.layers.28.self_attn.v_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.35.mlp.up_proj",
+    "model.language_model.layers.28.mlp.down_proj",
+    "model.language_model.layers.28.self_attn.k_proj",
+    "model.language_model.layers.30.self_attn.o_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.31.self_attn.o_proj",
+    "model.language_model.layers.32.self_attn.v_proj",
+    "model.language_model.layers.32.mlp.down_proj",
+    "model.language_model.layers.34.self_attn.v_proj",
+    "model.language_model.layers.34.self_attn.o_proj",
+    "model.language_model.layers.34.mlp.gate_proj",
+    "model.language_model.layers.30.mlp.up_proj",
+    "model.language_model.layers.35.self_attn.o_proj",
+    "model.language_model.layers.30.self_attn.v_proj",
+    "model.language_model.layers.33.self_attn.v_proj",
+    "model.language_model.layers.33.self_attn.k_proj",
+    "model.language_model.layers.33.self_attn.o_proj",
+    "model.language_model.layers.31.self_attn.v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Deep8_s42/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 42,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Deep8",
+    "layers": [
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35
+    ],
+    "ranks": {
+      "28": 72,
+      "29": 72,
+      "30": 72,
+      "31": 72,
+      "32": 72,
+      "33": 72,
+      "34": 72,
+      "35": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Deep8_s42/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 3.9035534858703613, "loss_scale": 4.0, "retries": 0, "seconds": 1.9, "max_memory_gb": 16.82, "ce": 0.6558216065168381, "kl": 0.11560958076734096, "context": 0.7136263996362686}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 2.8002266883850098, "loss_scale": 4.0, "retries": 0, "seconds": 3.1, "max_memory_gb": 17.16, "ce": 0.6083660274738065, "kl": 0.05966476090713968, "context": 0.6381984065303641}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 3.8795273303985596, "loss_scale": 4.0, "retries": 0, "seconds": 4.1, "max_memory_gb": 17.16, "ce": 0.6847236417233944, "kl": 0.03509709611535072, "context": 0.7022721879184246}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 3.481139659881592, "loss_scale": 4.0, "retries": 0, "seconds": 5.1, "max_memory_gb": 17.16, "ce": 0.6198926807701355, "kl": 0.6797756503801793, "context": 0.9597804778022692}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 2.54557466506958, "loss_scale": 4.0, "retries": 0, "seconds": 6.0, "max_memory_gb": 17.16, "ce": 0.34780728968326, "kl": 1.1837049000896513, "context": 0.9396597266895697}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 3.3097851276397705, "loss_scale": 4.0, "retries": 0, "seconds": 6.9, "max_memory_gb": 17.16, "ce": 0.4388646013685502, "kl": 0.3634428328368813, "context": 0.6205860138870776}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 3.6304967403411865, "loss_scale": 4.0, "retries": 0, "seconds": 7.9, "max_memory_gb": 17.16, "ce": 0.5953800231218338, "kl": 0.25627907377202064, "context": 0.723519578576088}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 0.46104326844215393, "loss_scale": 4.0, "retries": 0, "seconds": 8.8, "max_memory_gb": 17.16, "ce": 0.21414320718213276, "kl": 0.08747628704577437, "context": 0.2578813565260134}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 1.5892064571380615, "loss_scale": 4.0, "retries": 0, "seconds": 9.8, "max_memory_gb": 17.16, "ce": 0.5818650387227535, "kl": 0.12115913070738316, "context": 0.6424446105957031}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 5.366463661193848, "loss_scale": 4.0, "retries": 0, "seconds": 10.9, "max_memory_gb": 17.16, "ce": 0.5267138730851002, "kl": 0.3389527503750287, "context": 0.6961902445182204}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 1.1637603044509888, "loss_scale": 4.0, "retries": 0, "seconds": 11.9, "max_memory_gb": 17.16, "ce": 0.04728699973657058, "kl": 0.2949166549271922, "context": 0.19474533092306956}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 2.829620122909546, "loss_scale": 4.0, "retries": 0, "seconds": 12.9, "max_memory_gb": 17.16, "ce": 0.435618289076956, "kl": 1.21550249910797, "context": 1.043369532097131}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 1.1988158226013184, "loss_scale": 4.0, "retries": 0, "seconds": 13.9, "max_memory_gb": 17.16, "ce": 0.095564043847844, "kl": 0.05038636829704046, "context": 0.12075723125599325}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 1.762923240661621, "loss_scale": 4.0, "retries": 0, "seconds": 14.9, "max_memory_gb": 17.16, "ce": 0.07954805018380284, "kl": 0.1445939627592452, "context": 0.15184503188356757}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 2.9827539920806885, "loss_scale": 4.0, "retries": 0, "seconds": 16.0, "max_memory_gb": 17.16, "ce": 0.43757709220517427, "kl": 0.01175800763303414, "context": 0.44345610856544226}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 3.538012981414795, "loss_scale": 4.0, "retries": 0, "seconds": 17.0, "max_memory_gb": 17.16, "ce": 0.3605157564024921, "kl": 0.02893844027130399, "context": 0.37498496443095064}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 1.0155155658721924, "loss_scale": 4.0, "retries": 0, "seconds": 18.0, "max_memory_gb": 17.17, "ce": 0.016656835709000006, "kl": 0.10558374342508614, "context": 0.0694487064902205}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.39618349075317383, "loss_scale": 4.0, "retries": 0, "seconds": 19.1, "max_memory_gb": 17.17, "ce": 0.03849301557056606, "kl": 0.05419126420747489, "context": 0.06558864773251116}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.24527572095394135, "loss_scale": 4.0, "retries": 0, "seconds": 20.1, "max_memory_gb": 17.17, "ce": 0.00350157467983081, "kl": 0.013775130237718258, "context": 0.01038913993670576}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.505104124546051, "loss_scale": 4.0, "retries": 0, "seconds": 21.1, "max_memory_gb": 17.17, "ce": 0.00391058751847595, "kl": 0.0418965242588456, "context": 0.024858849777956493}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.3925231993198395, "loss_scale": 4.0, "retries": 0, "seconds": 22.2, "max_memory_gb": 17.17, "ce": 0.008384191198274493, "kl": 0.0929425046160759, "context": 0.05485544208204374}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.38145381212234497, "loss_scale": 4.0, "retries": 0, "seconds": 23.2, "max_memory_gb": 17.17, "ce": 0.014190926165611017, "kl": 0.009757604082551552, "context": 0.01906972855795175}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 2.5100793838500977, "loss_scale": 4.0, "retries": 0, "seconds": 24.2, "max_memory_gb": 17.17, "ce": 0.17003119055880234, "kl": 0.4882387764359919, "context": 0.41415057686390355}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 2.1126534938812256, "loss_scale": 4.0, "retries": 0, "seconds": 25.3, "max_memory_gb": 17.17, "ce": 0.04177619704569224, "kl": 0.6596828565379838, "context": 0.3716176137968432}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 3.4462907314300537, "loss_scale": 4.0, "retries": 0, "seconds": 26.3, "max_memory_gb": 17.17, "ce": 0.1501279730873648, "kl": 0.20249444540968398, "context": 0.25137518864357844}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 0.273355096578598, "loss_scale": 4.0, "retries": 0, "seconds": 27.4, "max_memory_gb": 17.17, "ce": 0.02449841300949629, "kl": 0.001384124379910645, "context": 0.025190474843839183}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 4.3090739250183105, "loss_scale": 4.0, "retries": 0, "seconds": 28.4, "max_memory_gb": 17.17, "ce": 0.4257746175862849, "kl": 0.02275828883284703, "context": 0.4371537652623374}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 3.478975534439087, "loss_scale": 4.0, "retries": 0, "seconds": 29.4, "max_memory_gb": 17.17, "ce": 0.014598234149161726, "kl": 0.8590118163265288, "context": 0.44410413317382336}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 0.9766550064086914, "loss_scale": 4.0, "retries": 0, "seconds": 30.5, "max_memory_gb": 17.17, "ce": 0.1164096433785744, "kl": 0.03564891393762082, "context": 0.13423410162795335}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 2.6167476177215576, "loss_scale": 4.0, "retries": 0, "seconds": 31.5, "max_memory_gb": 17.17, "ce": 0.0426380470598815, "kl": 0.6435944912373088, "context": 0.36443529289681464}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 2.317021369934082, "loss_scale": 4.0, "retries": 0, "seconds": 32.5, "max_memory_gb": 17.17, "ce": 0.49866707233013585, "kl": 0.15696259250398725, "context": 0.5771483774296939}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 0.47087010741233826, "loss_scale": 4.0, "retries": 0, "seconds": 33.6, "max_memory_gb": 17.17, "ce": 0.038100419798865914, "kl": 0.010217920411378145, "context": 0.04320938000455499}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.029021691530942917, "loss_scale": 4.0, "retries": 0, "seconds": 34.6, "max_memory_gb": 17.17, "ce": 0.0014660890446975827, "kl": 0.0013108857710903976, "context": 0.002121531892044004}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.057008206844329834, "loss_scale": 4.0, "retries": 0, "seconds": 35.5, "max_memory_gb": 17.17, "ce": 0.0016266992024611682, "kl": 0.0010320814917577081, "context": 0.0021427399769891053}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 1.3224713802337646, "loss_scale": 4.0, "retries": 0, "seconds": 36.7, "max_memory_gb": 17.17, "ce": 0.08128328766906634, "kl": 0.05388808964926284, "context": 0.10822733154054731}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 1.145294189453125, "loss_scale": 4.0, "retries": 0, "seconds": 37.6, "max_memory_gb": 17.17, "ce": 0.20737456240021857, "kl": 0.023000703582738424, "context": 0.2188749210035894}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 1.8414790630340576, "loss_scale": 4.0, "retries": 0, "seconds": 38.8, "max_memory_gb": 17.17, "ce": 0.03501607704674825, "kl": 0.47639194352086633, "context": 0.2732120563450735}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.7160285115242004, "loss_scale": 4.0, "retries": 0, "seconds": 39.7, "max_memory_gb": 17.17, "ce": 0.036796623360714875, "kl": 0.03697897517122328, "context": 0.055286109913140535}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 3.8743436336517334, "loss_scale": 4.0, "retries": 0, "seconds": 40.7, "max_memory_gb": 17.17, "ce": 0.2185823601903394, "kl": 0.19164497646852396, "context": 0.31440483895130455}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.4033258855342865, "loss_scale": 4.0, "retries": 0, "seconds": 41.8, "max_memory_gb": 17.17, "ce": 0.011923543755983701, "kl": 0.004642062491257093, "context": 0.014244575046177488}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 0.469732403755188, "loss_scale": 4.0, "retries": 0, "seconds": 42.8, "max_memory_gb": 17.17, "ce": 0.009621565623092465, "kl": 0.03748530050506815, "context": 0.028364216326735914}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 2.2627851963043213, "loss_scale": 4.0, "retries": 0, "seconds": 43.8, "max_memory_gb": 17.17, "ce": 0.010262208052154165, "kl": 0.44935525796609, "context": 0.23493983800290152}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 1.8097621202468872, "loss_scale": 4.0, "retries": 0, "seconds": 44.9, "max_memory_gb": 17.17, "ce": 0.11630816649994813, "kl": 0.16553940795711242, "context": 0.199077871395275}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 2.21175217628479, "loss_scale": 4.0, "retries": 0, "seconds": 45.9, "max_memory_gb": 17.17, "ce": 0.4699406801519217, "kl": 0.048410660558147356, "context": 0.49414601968601346}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 1.8696534633636475, "loss_scale": 4.0, "retries": 0, "seconds": 46.9, "max_memory_gb": 17.17, "ce": 0.335085139493458, "kl": 0.030082710647548083, "context": 0.35012648365227506}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 0.9383848309516907, "loss_scale": 4.0, "retries": 0, "seconds": 48.0, "max_memory_gb": 17.17, "ce": 0.25610666451393627, "kl": 0.031004241107439157, "context": 0.27160877487040125}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.09397338330745697, "loss_scale": 4.0, "retries": 0, "seconds": 49.0, "max_memory_gb": 17.17, "ce": 0.003204610708053224, "kl": 0.0026829107555386145, "context": 0.004546066222246736}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 1.046918511390686, "loss_scale": 4.0, "retries": 0, "seconds": 50.1, "max_memory_gb": 17.17, "ce": 0.1691927220672369, "kl": 0.02451821396516607, "context": 0.18145183008164167}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 3.7627127170562744, "loss_scale": 4.0, "retries": 0, "seconds": 51.1, "max_memory_gb": 17.17, "ce": 0.10665435597547912, "kl": 0.1301736397449531, "context": 0.17174118325056043}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 1.4171065092086792, "loss_scale": 4.0, "retries": 0, "seconds": 52.1, "max_memory_gb": 17.17, "ce": 0.017828226555138826, "kl": 0.14743880357127637, "context": 0.09154762723483145}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 2.330644130706787, "loss_scale": 4.0, "retries": 0, "seconds": 53.2, "max_memory_gb": 17.17, "ce": 0.10929016483714804, "kl": 0.23742539624799974, "context": 0.22800286917481571}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.25609737634658813, "loss_scale": 4.0, "retries": 0, "seconds": 54.2, "max_memory_gb": 17.17, "ce": 0.006913085373525973, "kl": 0.012972781717053294, "context": 0.013399476494669216}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 0.21657691895961761, "loss_scale": 4.0, "retries": 0, "seconds": 55.2, "max_memory_gb": 17.17, "ce": 0.011512463039252907, "kl": 0.029563174495706335, "context": 0.026294049806892872}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 0.6489869952201843, "loss_scale": 4.0, "retries": 0, "seconds": 56.3, "max_memory_gb": 17.17, "ce": 0.04750628570036497, "kl": 0.15506132882228485, "context": 0.12503694593033288}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 3.099449634552002, "loss_scale": 4.0, "retries": 0, "seconds": 57.3, "max_memory_gb": 17.17, "ce": 0.43086173614574363, "kl": 0.40634203855552187, "context": 0.6340327385405544}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.5823087692260742, "loss_scale": 4.0, "retries": 0, "seconds": 58.3, "max_memory_gb": 17.17, "ce": 0.021917953759839293, "kl": 0.06867418113597523, "context": 0.056255046190926805}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 1.0421921014785767, "loss_scale": 4.0, "retries": 0, "seconds": 59.4, "max_memory_gb": 17.17, "ce": 0.06512154071242549, "kl": 0.058710252051241696, "context": 0.09447666697087698}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 2.0218472480773926, "loss_scale": 4.0, "retries": 0, "seconds": 60.4, "max_memory_gb": 17.17, "ce": 0.017467454221332446, "kl": 0.5492737338645384, "context": 0.2921043161477428}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.11567912250757217, "loss_scale": 4.0, "retries": 0, "seconds": 61.5, "max_memory_gb": 17.17, "ce": 0.004756674497912172, "kl": 0.0007097502584656468, "context": 0.005111549602588639}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 0.7589711546897888, "loss_scale": 4.0, "retries": 0, "seconds": 62.4, "max_memory_gb": 17.17, "ce": 0.06112031461088918, "kl": 0.0641530029970454, "context": 0.09319681249326095}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 0.7969347238540649, "loss_scale": 4.0, "retries": 0, "seconds": 63.4, "max_memory_gb": 17.17, "ce": 0.005418521963292733, "kl": 0.10330092846561456, "context": 0.05706898507196456}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 1.875832438468933, "loss_scale": 4.0, "retries": 0, "seconds": 64.5, "max_memory_gb": 17.17, "ce": 0.04365344923280645, "kl": 0.2002197439869633, "context": 0.14376332249958068}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 1.1440147161483765, "loss_scale": 4.0, "retries": 0, "seconds": 65.5, "max_memory_gb": 17.17, "ce": 0.11829531694820616, "kl": 0.1478063697043126, "context": 0.1921985015651444}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.6605828404426575, "loss_scale": 4.0, "retries": 0, "seconds": 66.5, "max_memory_gb": 17.17, "ce": 0.045919552867417224, "kl": 0.07859522399508023, "context": 0.08521716351970099}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 2.1516213417053223, "loss_scale": 4.0, "retries": 0, "seconds": 67.6, "max_memory_gb": 17.17, "ce": 0.4597609137999825, "kl": 0.012201983176055364, "context": 0.46586189928348176}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 0.49353983998298645, "loss_scale": 4.0, "retries": 0, "seconds": 68.6, "max_memory_gb": 17.17, "ce": 0.03762363069108687, "kl": 0.025353562770760618, "context": 0.050300413931836374}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.335880309343338, "loss_scale": 4.0, "retries": 0, "seconds": 69.6, "max_memory_gb": 17.17, "ce": 0.010642829154676292, "kl": 0.0016548842186239199, "context": 0.011470271056168713}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.28347650170326233, "loss_scale": 4.0, "retries": 0, "seconds": 70.7, "max_memory_gb": 17.17, "ce": 0.028600279467354994, "kl": 0.013211634441177011, "context": 0.03520609719998902}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 1.5933691263198853, "loss_scale": 4.0, "retries": 0, "seconds": 71.7, "max_memory_gb": 17.17, "ce": 0.052209732559276745, "kl": 0.05576961033511907, "context": 0.08009454142302275}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 1.5832629203796387, "loss_scale": 4.0, "retries": 0, "seconds": 72.8, "max_memory_gb": 17.17, "ce": 0.3016824205114972, "kl": 0.06341954309209541, "context": 0.3333921902230941}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 1.8969097137451172, "loss_scale": 4.0, "retries": 0, "seconds": 73.8, "max_memory_gb": 17.17, "ce": 0.3778968175756745, "kl": 0.07112129000597633, "context": 0.4134574660565704}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.06112781539559364, "loss_scale": 4.0, "retries": 0, "seconds": 74.8, "max_memory_gb": 17.17, "ce": 0.002889910407247953, "kl": 0.006153069507490727, "context": 0.005966445052763447}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 1.347398042678833, "loss_scale": 4.0, "retries": 0, "seconds": 75.9, "max_memory_gb": 17.17, "ce": 0.34286977333249524, "kl": 0.009073337563677342, "context": 0.3474064337351592}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.31845998764038086, "loss_scale": 4.0, "retries": 0, "seconds": 76.9, "max_memory_gb": 17.17, "ce": 0.019780139366048388, "kl": 0.08024322066194145, "context": 0.059901747707044706}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 2.1843509674072266, "loss_scale": 4.0, "retries": 0, "seconds": 77.9, "max_memory_gb": 17.17, "ce": 0.1777039704611525, "kl": 0.27933771114658157, "context": 0.3173728271503933}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 2.019840717315674, "loss_scale": 4.0, "retries": 0, "seconds": 79.0, "max_memory_gb": 17.17, "ce": 0.007493015116779134, "kl": 0.39334634992701467, "context": 0.20416619061143138}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 1.1728503704071045, "loss_scale": 4.0, "retries": 0, "seconds": 80.0, "max_memory_gb": 17.17, "ce": 0.05153384403092787, "kl": 0.0729115956764872, "context": 0.08798964008747134}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 2.167577028274536, "loss_scale": 4.0, "retries": 0, "seconds": 80.9, "max_memory_gb": 17.17, "ce": 0.09611628722632304, "kl": 0.2780276322446298, "context": 0.2351300943410024}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.7509509325027466, "loss_scale": 4.0, "retries": 0, "seconds": 82.1, "max_memory_gb": 17.17, "ce": 0.08634142059599981, "kl": 0.1181014527392108, "context": 0.145392146660015}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.7877548933029175, "loss_scale": 4.0, "retries": 0, "seconds": 83.0, "max_memory_gb": 17.17, "ce": 0.21405930585751776, "kl": 0.018875057381023908, "context": 0.22349684200889897}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 1.5719956159591675, "loss_scale": 4.0, "retries": 0, "seconds": 84.1, "max_memory_gb": 17.17, "ce": 0.5350054235677817, "kl": 0.029970133415190503, "context": 0.549990495754173}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 2.117292881011963, "loss_scale": 4.0, "retries": 0, "seconds": 85.1, "max_memory_gb": 17.17, "ce": 0.0006568575190613046, "kl": 0.3109248002449476, "context": 0.15611926079873228}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.0228285789489746, "loss_scale": 4.0, "retries": 0, "seconds": 86.1, "max_memory_gb": 17.17, "ce": 0.06972321914508939, "kl": 0.0824898832506733, "context": 0.11096816044300795}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 0.8568670153617859, "loss_scale": 4.0, "retries": 0, "seconds": 87.2, "max_memory_gb": 17.17, "ce": 0.022198917984496802, "kl": 0.2984973930288106, "context": 0.17144761432427913}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.31172817945480347, "loss_scale": 4.0, "retries": 0, "seconds": 88.2, "max_memory_gb": 17.17, "ce": 0.034223784488858655, "kl": 0.05335685288082459, "context": 0.06090221095655579}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.43125784397125244, "loss_scale": 4.0, "retries": 0, "seconds": 89.2, "max_memory_gb": 17.17, "ce": 0.04412254970520735, "kl": 0.04123979921860155, "context": 0.06474245118442923}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.6088965535163879, "loss_scale": 4.0, "retries": 0, "seconds": 90.3, "max_memory_gb": 17.17, "ce": 0.025234055938199162, "kl": 0.020928637582983356, "context": 0.035698372856131755}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.44681382179260254, "loss_scale": 4.0, "retries": 0, "seconds": 91.2, "max_memory_gb": 17.17, "ce": 0.05231973360059783, "kl": 0.0016206453183258418, "context": 0.053130057494854555}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.23676274716854095, "loss_scale": 4.0, "retries": 0, "seconds": 92.2, "max_memory_gb": 17.17, "ce": 0.0037484575586859137, "kl": 0.023721210891380906, "context": 0.01560906259692274}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.013998436741530895, "loss_scale": 4.0, "retries": 0, "seconds": 93.3, "max_memory_gb": 17.17, "ce": 0.0006760860997019336, "kl": 0.001016238291413174, "context": 0.0011842052335850894}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 1.2220243215560913, "loss_scale": 4.0, "retries": 0, "seconds": 94.3, "max_memory_gb": 17.17, "ce": 0.05992758111824514, "kl": 0.0832230998785235, "context": 0.1015391310211271}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 1.8199462890625, "loss_scale": 4.0, "retries": 0, "seconds": 95.4, "max_memory_gb": 17.17, "ce": 0.14056211620481918, "kl": 0.019355653319507837, "context": 0.15023994515649974}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 1.9093270301818848, "loss_scale": 4.0, "retries": 0, "seconds": 96.4, "max_memory_gb": 17.17, "ce": 0.1328317997395061, "kl": 0.2844585549319163, "context": 0.27506106754299253}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 1.9994146823883057, "loss_scale": 4.0, "retries": 0, "seconds": 97.3, "max_memory_gb": 17.17, "ce": 0.38748509092692984, "kl": 0.32923875108826905, "context": 0.5521044659544714}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 1.5365524291992188, "loss_scale": 4.0, "retries": 0, "seconds": 98.4, "max_memory_gb": 17.17, "ce": 0.0907244321424514, "kl": 0.1613976899534464, "context": 0.1714232787489891}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 1.1491831541061401, "loss_scale": 4.0, "retries": 0, "seconds": 99.4, "max_memory_gb": 17.17, "ce": 0.008497170987538993, "kl": 0.21186729701730656, "context": 0.1144308217917569}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.39913707971572876, "loss_scale": 4.0, "retries": 0, "seconds": 100.4, "max_memory_gb": 17.17, "ce": 0.005614397479803301, "kl": 0.04127247783617349, "context": 0.026250637107295915}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.21375998854637146, "loss_scale": 4.0, "retries": 0, "seconds": 101.5, "max_memory_gb": 17.17, "ce": 0.023701792786596343, "kl": 0.017031540750394925, "context": 0.03221756230777828}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.5188719034194946, "loss_scale": 4.0, "retries": 0, "seconds": 102.5, "max_memory_gb": 17.17, "ce": 0.026330413078540005, "kl": 0.040802610443279264, "context": 0.046731718350201845}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.1547398567199707, "loss_scale": 4.0, "retries": 0, "seconds": 103.4, "max_memory_gb": 17.17, "ce": 0.013246496993815526, "kl": 0.008606375345152628, "context": 0.017549684780533426}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.5754351615905762, "loss_scale": 4.0, "retries": 0, "seconds": 104.5, "max_memory_gb": 17.17, "ce": 0.14275023230584338, "kl": 0.032743268757940314, "context": 0.15912186301648035}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.23710331320762634, "loss_scale": 4.0, "retries": 0, "seconds": 105.5, "max_memory_gb": 17.17, "ce": 0.0073701988367247395, "kl": 0.015004218253821477, "context": 0.014872307954647113}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.6729626059532166, "loss_scale": 4.0, "retries": 0, "seconds": 106.6, "max_memory_gb": 17.17, "ce": 0.19355726358480752, "kl": 0.06968838085595053, "context": 0.22840145858936012}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.5915459990501404, "loss_scale": 4.0, "retries": 0, "seconds": 107.6, "max_memory_gb": 17.17, "ce": 0.11802978802006692, "kl": 0.0035359154535399284, "context": 0.11979774665087461}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 1.206865906715393, "loss_scale": 4.0, "retries": 0, "seconds": 108.6, "max_memory_gb": 17.17, "ce": 0.1063350019576319, "kl": 0.24810073047882497, "context": 0.23038536696549272}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.8622151613235474, "loss_scale": 4.0, "retries": 0, "seconds": 109.7, "max_memory_gb": 17.17, "ce": 0.07335432760010008, "kl": 0.023955172024216154, "context": 0.08533191363676451}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.684211015701294, "loss_scale": 4.0, "retries": 0, "seconds": 110.6, "max_memory_gb": 17.17, "ce": 0.13087856359197758, "kl": 0.07039446220733225, "context": 0.1660757985082455}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 1.1271618604660034, "loss_scale": 4.0, "retries": 0, "seconds": 111.6, "max_memory_gb": 17.17, "ce": 0.26870662305736914, "kl": 0.002637078632687917, "context": 0.2700251577189192}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.8582795262336731, "loss_scale": 4.0, "retries": 0, "seconds": 112.7, "max_memory_gb": 17.17, "ce": 0.06851282261777669, "kl": 0.38459281904943055, "context": 0.26080922840628773}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.6669542193412781, "loss_scale": 4.0, "retries": 0, "seconds": 113.7, "max_memory_gb": 17.17, "ce": 0.05818675528280437, "kl": 0.009566253091179533, "context": 0.06296988189569674}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.03257614001631737, "loss_scale": 4.0, "retries": 0, "seconds": 114.7, "max_memory_gb": 17.17, "ce": 0.0008325845119543374, "kl": 0.0025652949741470366, "context": 0.0021152320041437633}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.7125474810600281, "loss_scale": 4.0, "retries": 0, "seconds": 115.8, "max_memory_gb": 17.17, "ce": 0.06973989144898951, "kl": 0.12304884796685656, "context": 0.13126431172713637}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 1.3081551790237427, "loss_scale": 4.0, "retries": 0, "seconds": 116.8, "max_memory_gb": 17.17, "ce": 0.5785571119959059, "kl": 0.11041378005393199, "context": 0.6337640020356048}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.1489705741405487, "loss_scale": 4.0, "retries": 0, "seconds": 117.9, "max_memory_gb": 17.17, "ce": 0.01547848604968749, "kl": 0.0006642810221819673, "context": 0.015810626311576925}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 1.6978247165679932, "loss_scale": 4.0, "retries": 0, "seconds": 118.8, "max_memory_gb": 17.17, "ce": 0.2778524705208838, "kl": 0.023326770577114075, "context": 0.2895158608444035}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 0.665996253490448, "loss_scale": 4.0, "retries": 0, "seconds": 119.8, "max_memory_gb": 17.17, "ce": 0.02520155147067271, "kl": 0.0019212209153920412, "context": 0.02616216184105724}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 1.1609588861465454, "loss_scale": 4.0, "retries": 0, "seconds": 120.9, "max_memory_gb": 17.17, "ce": 0.30474144017352955, "kl": 0.049197769210877595, "context": 0.32934032481716713}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 2.7018332481384277, "loss_scale": 4.0, "retries": 0, "seconds": 121.9, "max_memory_gb": 17.17, "ce": 0.15908167883753777, "kl": 0.9473806878777395, "context": 0.6327720085973851}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 2.152791738510132, "loss_scale": 4.0, "retries": 0, "seconds": 122.9, "max_memory_gb": 17.17, "ce": 0.13267125710262917, "kl": 0.0029245589594211197, "context": 0.13413353101350367}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 0.48073360323905945, "loss_scale": 4.0, "retries": 0, "seconds": 124.0, "max_memory_gb": 17.17, "ce": 0.02571694753714837, "kl": 0.049492449584704445, "context": 0.050463172323361505}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.44966086745262146, "loss_scale": 4.0, "retries": 0, "seconds": 125.0, "max_memory_gb": 17.17, "ce": 0.043254702919512056, "kl": 0.040440992751427984, "context": 0.06347519883274799}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.3722388744354248, "loss_scale": 4.0, "retries": 0, "seconds": 125.9, "max_memory_gb": 17.17, "ce": 0.00542087150097359, "kl": 0.15957714421165292, "context": 0.085209445504006}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.39372196793556213, "loss_scale": 4.0, "retries": 0, "seconds": 127.0, "max_memory_gb": 17.17, "ce": 0.015912370567093603, "kl": 0.0054845736540301004, "context": 0.018654656712897122}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 0.7941759824752808, "loss_scale": 4.0, "retries": 0, "seconds": 128.0, "max_memory_gb": 17.17, "ce": 0.03664462004962843, "kl": 0.6159514868929818, "context": 0.3446203560597496}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 1.4952064752578735, "loss_scale": 4.0, "retries": 0, "seconds": 129.1, "max_memory_gb": 17.17, "ce": 0.05386867836932652, "kl": 0.16659200848880573, "context": 0.13716468354687095}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 1.5348445177078247, "loss_scale": 4.0, "retries": 0, "seconds": 130.1, "max_memory_gb": 17.17, "ce": 0.5970403361134231, "kl": 0.2085484642660731, "context": 0.7013145904638804}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.2120622843503952, "loss_scale": 4.0, "retries": 0, "seconds": 131.1, "max_memory_gb": 17.17, "ce": 0.0025304424634668976, "kl": 0.038983446913334774, "context": 0.02202216614386998}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.38122209906578064, "loss_scale": 4.0, "retries": 0, "seconds": 132.2, "max_memory_gb": 17.17, "ce": 0.046482018326059915, "kl": 0.0392089762899559, "context": 0.0660865078389179}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 1.2772842645645142, "loss_scale": 4.0, "retries": 0, "seconds": 133.2, "max_memory_gb": 17.17, "ce": 0.20574555778875947, "kl": 0.08581846022389072, "context": 0.2486547879234422}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.5114309191703796, "loss_scale": 4.0, "retries": 0, "seconds": 134.1, "max_memory_gb": 17.17, "ce": 0.039902715070638806, "kl": 0.06222818430978805, "context": 0.07101680722553283}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 0.45810914039611816, "loss_scale": 4.0, "retries": 0, "seconds": 135.2, "max_memory_gb": 17.17, "ce": 0.16498647240223363, "kl": 0.07628985771771113, "context": 0.20313139993231744}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.024873115122318268, "loss_scale": 4.0, "retries": 0, "seconds": 136.2, "max_memory_gb": 17.17, "ce": 0.00229105404287111, "kl": 9.278219795305631e-05, "context": 0.0023374451411655173}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.09084931015968323, "loss_scale": 4.0, "retries": 0, "seconds": 137.2, "max_memory_gb": 17.17, "ce": 0.0110678665296291, "kl": 0.0018848802574211732, "context": 0.012010306949377991}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 1.545754313468933, "loss_scale": 4.0, "retries": 0, "seconds": 138.3, "max_memory_gb": 17.17, "ce": 0.045064352816552855, "kl": 0.08749897685157748, "context": 0.08881383707921486}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 1.2919485569000244, "loss_scale": 4.0, "retries": 0, "seconds": 139.3, "max_memory_gb": 17.17, "ce": 0.32168408104917035, "kl": 0.46329564915504307, "context": 0.5533319240785204}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 0.11042992770671844, "loss_scale": 4.0, "retries": 0, "seconds": 140.3, "max_memory_gb": 17.17, "ce": 0.010243737386190332, "kl": 0.006599498330615461, "context": 0.013543486420530826}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 2.7831404209136963, "loss_scale": 4.0, "retries": 0, "seconds": 141.2, "max_memory_gb": 17.17, "ce": 0.4157399415562395, "kl": 0.26167454887036, "context": 0.5465772156312596}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.1428060531616211, "loss_scale": 4.0, "retries": 0, "seconds": 142.1, "max_memory_gb": 17.17, "ce": 0.015120575175387785, "kl": 0.0056990662415046245, "context": 0.017970108485314995}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 1.2336182594299316, "loss_scale": 4.0, "retries": 0, "seconds": 143.1, "max_memory_gb": 17.17, "ce": 0.3507224323111586, "kl": 0.008569134170102188, "context": 0.35500698938267305}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.4257831871509552, "loss_scale": 4.0, "retries": 0, "seconds": 144.0, "max_memory_gb": 17.17, "ce": 0.020030664338264614, "kl": 0.008434787578153191, "context": 0.0242480592278298}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 3.016634225845337, "loss_scale": 4.0, "retries": 0, "seconds": 145.0, "max_memory_gb": 17.17, "ce": 0.21137082163477316, "kl": 0.2689971663057804, "context": 0.3458694047294557}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 1.3438292741775513, "loss_scale": 4.0, "retries": 0, "seconds": 146.1, "max_memory_gb": 17.17, "ce": 0.0031560398056171834, "kl": 0.17232689482625574, "context": 0.08931948625831865}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 0.4508211314678192, "loss_scale": 4.0, "retries": 0, "seconds": 147.0, "max_memory_gb": 17.17, "ce": 0.1503128191397991, "kl": 0.10337762953713536, "context": 0.20200163457775488}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 1.269636631011963, "loss_scale": 4.0, "retries": 0, "seconds": 148.0, "max_memory_gb": 17.17, "ce": 0.0037156126491026953, "kl": 0.14738683329778723, "context": 0.07740902830846608}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.22705627977848053, "loss_scale": 4.0, "retries": 0, "seconds": 149.0, "max_memory_gb": 17.17, "ce": 0.020101334957871586, "kl": 0.005727108435166883, "context": 0.022964889052673243}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.03862845525145531, "loss_scale": 4.0, "retries": 0, "seconds": 149.9, "max_memory_gb": 17.17, "ce": 0.0027045557217206806, "kl": 0.0018217817341792397, "context": 0.003615446505136788}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 0.4797735810279846, "loss_scale": 4.0, "retries": 0, "seconds": 150.9, "max_memory_gb": 17.17, "ce": 0.08686946405214258, "kl": 0.00297239440260455, "context": 0.08835566104971804}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 2.4282352924346924, "loss_scale": 4.0, "retries": 0, "seconds": 151.8, "max_memory_gb": 17.17, "ce": 0.33118941268185154, "kl": 0.20087083269027062, "context": 0.4316248127142899}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 2.34830379486084, "loss_scale": 4.0, "retries": 0, "seconds": 152.7, "max_memory_gb": 17.17, "ce": 0.1340147740993416, "kl": 0.23523679497702688, "context": 0.2516331756778527}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.9990683197975159, "loss_scale": 4.0, "retries": 0, "seconds": 153.7, "max_memory_gb": 17.17, "ce": 0.003037200032849796, "kl": 0.4786089227645789, "context": 0.24234165511734318}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.05241882801055908, "loss_scale": 4.0, "retries": 0, "seconds": 154.6, "max_memory_gb": 17.17, "ce": 0.0046777521492913365, "kl": 0.001032916028634645, "context": 0.0051942100981250405}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 0.6079447269439697, "loss_scale": 4.0, "retries": 0, "seconds": 155.5, "max_memory_gb": 17.17, "ce": 0.0829616082191933, "kl": 0.011679783376166597, "context": 0.08880149622564204}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.11230204254388809, "loss_scale": 4.0, "retries": 0, "seconds": 156.5, "max_memory_gb": 17.17, "ce": 0.005889734617085196, "kl": 0.01166584057500586, "context": 0.011722654802724719}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.10181357711553574, "loss_scale": 4.0, "retries": 0, "seconds": 157.3, "max_memory_gb": 17.17, "ce": 0.011645977276202757, "kl": 0.0013484537198564794, "context": 0.012320204048592132}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.38782796263694763, "loss_scale": 4.0, "retries": 0, "seconds": 158.3, "max_memory_gb": 17.17, "ce": 0.10790972723043524, "kl": 0.04573490134816893, "context": 0.13077717230771668}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.7358171343803406, "loss_scale": 4.0, "retries": 0, "seconds": 159.3, "max_memory_gb": 17.17, "ce": 0.11655350824003108, "kl": 0.009252265346731292, "context": 0.12117964151548222}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.05833827331662178, "loss_scale": 4.0, "retries": 0, "seconds": 160.1, "max_memory_gb": 17.17, "ce": 0.002622548272483982, "kl": 0.0018354562598688062, "context": 0.003540276433341205}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.3860770463943481, "loss_scale": 4.0, "retries": 0, "seconds": 161.1, "max_memory_gb": 17.17, "ce": 0.0059556407504715025, "kl": 0.44889362626418006, "context": 0.23040245875017717}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.8530033826828003, "loss_scale": 4.0, "retries": 0, "seconds": 162.0, "max_memory_gb": 17.17, "ce": 0.030141681840177625, "kl": 0.051972372448290116, "context": 0.05612786806887016}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.5930055379867554, "loss_scale": 4.0, "retries": 0, "seconds": 162.9, "max_memory_gb": 17.17, "ce": 0.050966973714821506, "kl": 0.030202079206901544, "context": 0.06606801395537332}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 0.41593655943870544, "loss_scale": 4.0, "retries": 0, "seconds": 163.9, "max_memory_gb": 17.17, "ce": 0.07000558386789635, "kl": 0.03943657362833619, "context": 0.08972386928508058}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 0.39973148703575134, "loss_scale": 4.0, "retries": 0, "seconds": 164.8, "max_memory_gb": 17.17, "ce": 0.020634121072362177, "kl": 0.06659631121146958, "context": 0.05393227666354505}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 0.4043184816837311, "loss_scale": 4.0, "retries": 0, "seconds": 165.7, "max_memory_gb": 17.17, "ce": 0.08648134308168665, "kl": 0.05775055849335331, "context": 0.11535662232199684}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.5714597702026367, "loss_scale": 4.0, "retries": 0, "seconds": 166.7, "max_memory_gb": 17.17, "ce": 0.15003579383483157, "kl": 0.027106780151370913, "context": 0.16358918300829828}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.2814876437187195, "loss_scale": 4.0, "retries": 0, "seconds": 167.6, "max_memory_gb": 17.17, "ce": 0.00274127593729645, "kl": 0.029494578752292, "context": 0.017488565645180643}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.15362872183322906, "loss_scale": 4.0, "retries": 0, "seconds": 168.5, "max_memory_gb": 17.17, "ce": 0.014522571815177798, "kl": 0.017423396922822576, "context": 0.023234270280227065}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.18086080253124237, "loss_scale": 4.0, "retries": 0, "seconds": 169.5, "max_memory_gb": 17.17, "ce": 0.007940329720440786, "kl": 0.0030131534640531754, "context": 0.009446906442462932}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.804979145526886, "loss_scale": 4.0, "retries": 0, "seconds": 170.4, "max_memory_gb": 17.17, "ce": 0.018591916596051306, "kl": 0.11397574887087103, "context": 0.0755797945894301}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 0.3435262143611908, "loss_scale": 4.0, "retries": 0, "seconds": 171.4, "max_memory_gb": 17.17, "ce": 0.02587968739680946, "kl": 0.13073832455847878, "context": 0.09124885153141804}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.9228337407112122, "loss_scale": 4.0, "retries": 0, "seconds": 172.3, "max_memory_gb": 17.17, "ce": 0.06889646759373136, "kl": 0.17538511804741574, "context": 0.15658902656286955}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.8133348822593689, "loss_scale": 4.0, "retries": 0, "seconds": 173.2, "max_memory_gb": 17.17, "ce": 0.15668174935126444, "kl": 0.013288481506606331, "context": 0.16332599382440094}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.9741442203521729, "loss_scale": 4.0, "retries": 0, "seconds": 174.2, "max_memory_gb": 17.17, "ce": 0.19362597446888685, "kl": 0.025960512546589598, "context": 0.20660623151343316}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.643818199634552, "loss_scale": 4.0, "retries": 0, "seconds": 175.1, "max_memory_gb": 17.17, "ce": 0.03479617890843656, "kl": 0.044736893934896216, "context": 0.057164625817677006}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.760993242263794, "loss_scale": 4.0, "retries": 0, "seconds": 175.9, "max_memory_gb": 17.17, "ce": 0.10022546019172296, "kl": 0.06893053138628602, "context": 0.13469073333544657}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 0.36581265926361084, "loss_scale": 4.0, "retries": 0, "seconds": 176.9, "max_memory_gb": 17.17, "ce": 0.026051132997963578, "kl": 0.00956686590791378, "context": 0.03083456671447493}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.3678165674209595, "loss_scale": 4.0, "retries": 0, "seconds": 177.8, "max_memory_gb": 17.17, "ce": 0.012189363595098257, "kl": 0.24952609431784367, "context": 0.1369524101028219}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 2.2008979320526123, "loss_scale": 4.0, "retries": 0, "seconds": 178.7, "max_memory_gb": 17.17, "ce": 0.24654447444481775, "kl": 0.11962601153936703, "context": 0.30635748972417787}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.7966410517692566, "loss_scale": 4.0, "retries": 0, "seconds": 179.7, "max_memory_gb": 17.17, "ce": 0.001849619162385352, "kl": 0.21079091335923295, "context": 0.10724507889244705}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 0.39352741837501526, "loss_scale": 4.0, "retries": 0, "seconds": 180.6, "max_memory_gb": 17.17, "ce": 0.08652513028937392, "kl": 0.08062548883208365, "context": 0.12683787377318367}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.5766645669937134, "loss_scale": 4.0, "retries": 0, "seconds": 181.6, "max_memory_gb": 17.17, "ce": 0.22745670239964966, "kl": 0.013378305449805339, "context": 0.23414586037688423}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 1.2833775281906128, "loss_scale": 4.0, "retries": 0, "seconds": 182.5, "max_memory_gb": 17.17, "ce": 0.32860351220006123, "kl": 0.1379155475005973, "context": 0.39756128448061645}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 1.26497483253479, "loss_scale": 4.0, "retries": 0, "seconds": 183.4, "max_memory_gb": 17.17, "ce": 0.1449785807635635, "kl": 0.1149477533181198, "context": 0.20245245602563955}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.037863120436668396, "loss_scale": 4.0, "retries": 0, "seconds": 184.4, "max_memory_gb": 17.17, "ce": 0.0014694128767587245, "kl": 0.0028618553624255583, "context": 0.002900340565247461}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.023493345826864243, "loss_scale": 4.0, "retries": 0, "seconds": 185.4, "max_memory_gb": 17.17, "ce": 0.0019394326373003423, "kl": 0.0009562202612869442, "context": 0.002417542797047645}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.044191308319568634, "loss_scale": 4.0, "retries": 0, "seconds": 186.3, "max_memory_gb": 17.17, "ce": 0.0025360613217344508, "kl": 0.0015454925742233172, "context": 0.0033088076452258974}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 0.16300630569458008, "loss_scale": 4.0, "retries": 0, "seconds": 187.3, "max_memory_gb": 17.17, "ce": 0.004943735068081878, "kl": 0.016477152763400227, "context": 0.01318231166806072}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 0.69459468126297, "loss_scale": 4.0, "retries": 0, "seconds": 188.3, "max_memory_gb": 17.17, "ce": 0.16324632294708863, "kl": 0.004112450718821492, "context": 0.16530255414545536}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 0.42978182435035706, "loss_scale": 4.0, "retries": 0, "seconds": 189.3, "max_memory_gb": 17.17, "ce": 0.05644015266443603, "kl": 0.1162114024082257, "context": 0.11454585660248995}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 0.5212754011154175, "loss_scale": 4.0, "retries": 0, "seconds": 190.5, "max_memory_gb": 17.17, "ce": 0.10600827832240611, "kl": 0.012346955640850865, "context": 0.11218175389512908}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.8455212116241455, "loss_scale": 4.0, "retries": 0, "seconds": 191.5, "max_memory_gb": 17.17, "ce": 0.03672196087427437, "kl": 0.017765563970897347, "context": 0.04560474102618173}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.05695787072181702, "loss_scale": 4.0, "retries": 0, "seconds": 192.6, "max_memory_gb": 17.17, "ce": 0.00409081790712662, "kl": 0.003726676670339657, "context": 0.00595415620773565}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 2.576688051223755, "loss_scale": 4.0, "retries": 0, "seconds": 193.6, "max_memory_gb": 17.17, "ce": 0.4319506098108832, "kl": 0.3752718006289797, "context": 0.6195865173940547}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.7887452244758606, "loss_scale": 4.0, "retries": 0, "seconds": 194.6, "max_memory_gb": 17.17, "ce": 0.05060849440633319, "kl": 0.1078214742410637, "context": 0.1045192305900855}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.7634215354919434, "loss_scale": 4.0, "retries": 0, "seconds": 195.7, "max_memory_gb": 17.17, "ce": 0.004523246054304764, "kl": 0.1152844697207911, "context": 0.06216548086376861}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.034857865422964096, "loss_scale": 4.0, "retries": 0, "seconds": 196.7, "max_memory_gb": 17.17, "ce": 0.0026522442931309342, "kl": 0.0005617017886834219, "context": 0.002933095267508179}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.05158420652151108, "loss_scale": 4.0, "retries": 0, "seconds": 197.7, "max_memory_gb": 17.17, "ce": 0.0028948147082701325, "kl": 0.0007773097657661765, "context": 0.003283469646703452}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.11432158946990967, "loss_scale": 4.0, "retries": 0, "seconds": 198.8, "max_memory_gb": 17.17, "ce": 0.0010107098423759453, "kl": 0.00861027381324675, "context": 0.0053158468144829385}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.8117381930351257, "loss_scale": 4.0, "retries": 0, "seconds": 199.8, "max_memory_gb": 17.17, "ce": 0.0028707406454486772, "kl": 0.26563034874925506, "context": 0.13568591488001402}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 0.8719722032546997, "loss_scale": 4.0, "retries": 0, "seconds": 200.8, "max_memory_gb": 17.17, "ce": 0.09383138797420543, "kl": 0.23113001618912676, "context": 0.2093963990919292}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.08443251252174377, "loss_scale": 4.0, "retries": 0, "seconds": 201.9, "max_memory_gb": 17.17, "ce": 0.011183450595126487, "kl": 0.0011286416847724468, "context": 0.011747771728551015}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.40546858310699463, "loss_scale": 4.0, "retries": 0, "seconds": 202.9, "max_memory_gb": 17.17, "ce": 0.030478220724035054, "kl": 0.020796815573703498, "context": 0.040876628598198295}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 1.0983099937438965, "loss_scale": 4.0, "retries": 0, "seconds": 204.1, "max_memory_gb": 17.17, "ce": 0.017112704808823764, "kl": 1.0081440044450574, "context": 0.5211847013561055}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.1745569407939911, "loss_scale": 4.0, "retries": 0, "seconds": 205.1, "max_memory_gb": 17.17, "ce": 0.014378269203007221, "kl": 0.010215471549599897, "context": 0.019486005694488995}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.06519432365894318, "loss_scale": 4.0, "retries": 0, "seconds": 206.1, "max_memory_gb": 17.17, "ce": 0.00604699325049296, "kl": 0.001833615835494129, "context": 0.006963801104575396}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.1369883418083191, "loss_scale": 4.0, "retries": 0, "seconds": 207.2, "max_memory_gb": 17.17, "ce": 0.014366236631758511, "kl": 0.0070174154825508595, "context": 0.01787494495511055}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.7136102914810181, "loss_scale": 4.0, "retries": 0, "seconds": 208.1, "max_memory_gb": 17.17, "ce": 0.028136113716755062, "kl": 0.04275651622447185, "context": 0.04951437271665782}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.22996735572814941, "loss_scale": 4.0, "retries": 0, "seconds": 209.1, "max_memory_gb": 17.17, "ce": 0.016051385187893175, "kl": 0.005087003523840394, "context": 0.01859488735499326}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.5232735276222229, "loss_scale": 4.0, "retries": 0, "seconds": 210.2, "max_memory_gb": 17.17, "ce": 0.023137405514717102, "kl": 0.017497744600404985, "context": 0.03188627731287852}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.320197731256485, "loss_scale": 4.0, "retries": 0, "seconds": 211.2, "max_memory_gb": 17.17, "ce": 0.039645152442972176, "kl": 0.014172537994454615, "context": 0.046731422597076744}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 0.48997199535369873, "loss_scale": 4.0, "retries": 0, "seconds": 212.2, "max_memory_gb": 17.17, "ce": 0.08264218652038835, "kl": 0.0276451052777702, "context": 0.09646473848260939}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.345649391412735, "loss_scale": 4.0, "retries": 0, "seconds": 213.4, "max_memory_gb": 17.17, "ce": 0.0112039293453563, "kl": 0.01496379910986434, "context": 0.018685828428715467}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 0.5594438910484314, "loss_scale": 4.0, "retries": 0, "seconds": 214.3, "max_memory_gb": 17.17, "ce": 0.12251370315789245, "kl": 0.2009116836125031, "context": 0.2229695375135634}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 1.0285007953643799, "loss_scale": 4.0, "retries": 0, "seconds": 215.5, "max_memory_gb": 17.17, "ce": 0.33762180496705696, "kl": 0.5411519223853247, "context": 0.608197766385274}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.18013904988765717, "loss_scale": 4.0, "retries": 0, "seconds": 216.5, "max_memory_gb": 17.17, "ce": 0.021321210777387023, "kl": 0.0036078345656278543, "context": 0.023125127743696794}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.4923410713672638, "loss_scale": 4.0, "retries": 0, "seconds": 217.4, "max_memory_gb": 17.17, "ce": 0.03827136306790635, "kl": 0.012117277830839157, "context": 0.044330001110211015}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.7926775217056274, "loss_scale": 4.0, "retries": 0, "seconds": 218.6, "max_memory_gb": 17.17, "ce": 0.004126985892071389, "kl": 0.13701891566597624, "context": 0.07263644045451656}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 1.3708926439285278, "loss_scale": 4.0, "retries": 0, "seconds": 219.6, "max_memory_gb": 17.17, "ce": 0.0015009701273811515, "kl": 0.37275962448620703, "context": 0.18788078795842011}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 3.899045467376709, "loss_scale": 4.0, "retries": 0, "seconds": 220.5, "max_memory_gb": 17.17, "ce": 0.16781036247266456, "kl": 1.1557321937580127, "context": 0.7456764721428044}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.17286770045757294, "loss_scale": 4.0, "retries": 0, "seconds": 221.7, "max_memory_gb": 17.17, "ce": 0.015206038151518442, "kl": 0.03581488521513165, "context": 0.03311347994895186}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 1.4135265350341797, "loss_scale": 4.0, "retries": 0, "seconds": 222.7, "max_memory_gb": 17.17, "ce": 0.11362784932134673, "kl": 0.360343167974861, "context": 0.2937994179665111}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 1.2727798223495483, "loss_scale": 4.0, "retries": 0, "seconds": 223.7, "max_memory_gb": 17.17, "ce": 0.23467661961331032, "kl": 0.03574129432672635, "context": 0.25254727236460894}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.3992897570133209, "loss_scale": 4.0, "retries": 0, "seconds": 224.8, "max_memory_gb": 17.17, "ce": 0.010537035777815618, "kl": 0.06782692197884899, "context": 0.044450498098740354}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 0.6070038676261902, "loss_scale": 4.0, "retries": 0, "seconds": 225.8, "max_memory_gb": 17.17, "ce": 0.012476008443627506, "kl": 0.10323059500660747, "context": 0.06409130780957639}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.01331840455532074, "loss_scale": 4.0, "retries": 0, "seconds": 226.8, "max_memory_gb": 17.17, "ce": 0.0010664186520443764, "kl": 0.0001018860819215206, "context": 0.0011173616803716868}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.9690026044845581, "loss_scale": 4.0, "retries": 0, "seconds": 227.8, "max_memory_gb": 17.17, "ce": 0.0252594436678919, "kl": 0.7898293980820199, "context": 0.4201741533615859}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.30196651816368103, "loss_scale": 4.0, "retries": 0, "seconds": 228.8, "max_memory_gb": 17.17, "ce": 0.07127650501206517, "kl": 0.14821980148553848, "context": 0.14538640901446342}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.18908017873764038, "loss_scale": 4.0, "retries": 0, "seconds": 229.9, "max_memory_gb": 17.17, "ce": 0.01088306400924921, "kl": 0.023294647020520642, "context": 0.0225303876795806}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 1.9184678792953491, "loss_scale": 4.0, "retries": 0, "seconds": 230.9, "max_memory_gb": 17.17, "ce": 0.11139805254060775, "kl": 0.133307454077908, "context": 0.1780517870793119}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 1.007952094078064, "loss_scale": 4.0, "retries": 0, "seconds": 231.9, "max_memory_gb": 17.17, "ce": 0.4493102957494557, "kl": 0.014294068329036236, "context": 0.4564573303796351}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.2873489260673523, "loss_scale": 4.0, "retries": 0, "seconds": 233.0, "max_memory_gb": 17.17, "ce": 0.003816623575403355, "kl": 0.031156540564552415, "context": 0.019394894101424143}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.31371399760246277, "loss_scale": 4.0, "retries": 0, "seconds": 234.0, "max_memory_gb": 17.17, "ce": 0.06520618191279937, "kl": 0.22833739344787318, "context": 0.17937487526796758}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.6022036671638489, "loss_scale": 4.0, "retries": 0, "seconds": 235.0, "max_memory_gb": 17.17, "ce": 0.0019826806092169136, "kl": 0.06494516489328817, "context": 0.03445526283758227}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.3118502199649811, "loss_scale": 4.0, "retries": 0, "seconds": 236.1, "max_memory_gb": 17.17, "ce": 0.05103295887238346, "kl": 0.004542111948467209, "context": 0.05330401464016177}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.7104184031486511, "loss_scale": 4.0, "retries": 0, "seconds": 237.1, "max_memory_gb": 17.17, "ce": 0.05991029666620307, "kl": 0.06826180472853594, "context": 0.09404119895771146}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 0.6784074902534485, "loss_scale": 4.0, "retries": 0, "seconds": 238.2, "max_memory_gb": 17.17, "ce": 0.06245629051409196, "kl": 0.0655896085590939, "context": 0.09525109484093264}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.4516635835170746, "loss_scale": 4.0, "retries": 0, "seconds": 239.2, "max_memory_gb": 17.17, "ce": 0.07578341269982047, "kl": 0.020902972042676993, "context": 0.08623489993624389}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 1.4584685564041138, "loss_scale": 4.0, "retries": 0, "seconds": 240.2, "max_memory_gb": 17.17, "ce": 0.02574781747534871, "kl": 0.7724295546940994, "context": 0.4119625882594846}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 1.8516544103622437, "loss_scale": 4.0, "retries": 0, "seconds": 241.3, "max_memory_gb": 17.17, "ce": 0.2694474149902817, "kl": 0.1408120661508292, "context": 0.3398534332518466}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.10606920719146729, "loss_scale": 4.0, "retries": 0, "seconds": 242.3, "max_memory_gb": 17.17, "ce": 0.0069676187922596, "kl": 0.000997071729216259, "context": 0.007466154842404649}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 1.1372952461242676, "loss_scale": 4.0, "retries": 0, "seconds": 243.2, "max_memory_gb": 17.17, "ce": 0.23880948685109615, "kl": 0.1284986500540981, "context": 0.30305881425738335}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.49746671319007874, "loss_scale": 4.0, "retries": 0, "seconds": 244.4, "max_memory_gb": 17.17, "ce": 0.019276947728940286, "kl": 0.09018520489189541, "context": 0.06436955145909451}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 1.5548676252365112, "loss_scale": 4.0, "retries": 0, "seconds": 245.4, "max_memory_gb": 17.17, "ce": 0.2861197356687626, "kl": 0.07598660378425848, "context": 0.3241130482638255}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.9816688895225525, "loss_scale": 4.0, "retries": 0, "seconds": 246.4, "max_memory_gb": 17.17, "ce": 0.19278571981703863, "kl": 0.03746531127762864, "context": 0.21151837997604162}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 0.4770619869232178, "loss_scale": 4.0, "retries": 0, "seconds": 247.5, "max_memory_gb": 17.17, "ce": 0.027150997426360846, "kl": 0.09650489128944173, "context": 0.07540344051085413}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.7330482602119446, "loss_scale": 4.0, "retries": 0, "seconds": 248.4, "max_memory_gb": 17.17, "ce": 0.10686133844865253, "kl": 0.2866552911527833, "context": 0.2501889943014248}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.457785964012146, "loss_scale": 4.0, "retries": 0, "seconds": 249.6, "max_memory_gb": 17.17, "ce": 0.037877033930271864, "kl": 0.06698427390074357, "context": 0.07136916881427169}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.03300483524799347, "loss_scale": 4.0, "retries": 0, "seconds": 250.5, "max_memory_gb": 17.17, "ce": 0.0032519517990294844, "kl": 0.0005721560228266753, "context": 0.003538029792252928}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.03142354637384415, "loss_scale": 4.0, "retries": 0, "seconds": 251.5, "max_memory_gb": 17.17, "ce": 0.0031111329735722393, "kl": 0.0002671759375516558, "context": 0.0032447209378005937}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.19506417214870453, "loss_scale": 4.0, "retries": 0, "seconds": 252.6, "max_memory_gb": 17.17, "ce": 0.016595691107795574, "kl": 0.0044418389443308115, "context": 0.018816610085195862}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.02134028449654579, "loss_scale": 4.0, "retries": 0, "seconds": 253.6, "max_memory_gb": 17.17, "ce": 0.0017662633617874235, "kl": 0.0009437776180902802, "context": 0.0022381522139767185}

runs/fixed_internvl/context/Deep8_s73/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Deep8_s73/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.28.mlp.down_proj": 144,
+    "model.language_model.layers.28.mlp.gate_proj": 144,
+    "model.language_model.layers.28.mlp.up_proj": 144,
+    "model.language_model.layers.28.self_attn.k_proj": 144,
+    "model.language_model.layers.28.self_attn.o_proj": 144,
+    "model.language_model.layers.28.self_attn.q_proj": 144,
+    "model.language_model.layers.28.self_attn.v_proj": 144,
+    "model.language_model.layers.29.mlp.down_proj": 144,
+    "model.language_model.layers.29.mlp.gate_proj": 144,
+    "model.language_model.layers.29.mlp.up_proj": 144,
+    "model.language_model.layers.29.self_attn.k_proj": 144,
+    "model.language_model.layers.29.self_attn.o_proj": 144,
+    "model.language_model.layers.29.self_attn.q_proj": 144,
+    "model.language_model.layers.29.self_attn.v_proj": 144,
+    "model.language_model.layers.30.mlp.down_proj": 144,
+    "model.language_model.layers.30.mlp.gate_proj": 144,
+    "model.language_model.layers.30.mlp.up_proj": 144,
+    "model.language_model.layers.30.self_attn.k_proj": 144,
+    "model.language_model.layers.30.self_attn.o_proj": 144,
+    "model.language_model.layers.30.self_attn.q_proj": 144,
+    "model.language_model.layers.30.self_attn.v_proj": 144,
+    "model.language_model.layers.31.mlp.down_proj": 144,
+    "model.language_model.layers.31.mlp.gate_proj": 144,
+    "model.language_model.layers.31.mlp.up_proj": 144,
+    "model.language_model.layers.31.self_attn.k_proj": 144,
+    "model.language_model.layers.31.self_attn.o_proj": 144,
+    "model.language_model.layers.31.self_attn.q_proj": 144,
+    "model.language_model.layers.31.self_attn.v_proj": 144,
+    "model.language_model.layers.32.mlp.down_proj": 144,
+    "model.language_model.layers.32.mlp.gate_proj": 144,
+    "model.language_model.layers.32.mlp.up_proj": 144,
+    "model.language_model.layers.32.self_attn.k_proj": 144,
+    "model.language_model.layers.32.self_attn.o_proj": 144,
+    "model.language_model.layers.32.self_attn.q_proj": 144,
+    "model.language_model.layers.32.self_attn.v_proj": 144,
+    "model.language_model.layers.33.mlp.down_proj": 144,
+    "model.language_model.layers.33.mlp.gate_proj": 144,
+    "model.language_model.layers.33.mlp.up_proj": 144,
+    "model.language_model.layers.33.self_attn.k_proj": 144,
+    "model.language_model.layers.33.self_attn.o_proj": 144,
+    "model.language_model.layers.33.self_attn.q_proj": 144,
+    "model.language_model.layers.33.self_attn.v_proj": 144,
+    "model.language_model.layers.34.mlp.down_proj": 144,
+    "model.language_model.layers.34.mlp.gate_proj": 144,
+    "model.language_model.layers.34.mlp.up_proj": 144,
+    "model.language_model.layers.34.self_attn.k_proj": 144,
+    "model.language_model.layers.34.self_attn.o_proj": 144,
+    "model.language_model.layers.34.self_attn.q_proj": 144,
+    "model.language_model.layers.34.self_attn.v_proj": 144,
+    "model.language_model.layers.35.mlp.down_proj": 144,
+    "model.language_model.layers.35.mlp.gate_proj": 144,
+    "model.language_model.layers.35.mlp.up_proj": 144,
+    "model.language_model.layers.35.self_attn.k_proj": 144,
+    "model.language_model.layers.35.self_attn.o_proj": 144,
+    "model.language_model.layers.35.self_attn.q_proj": 144,
+    "model.language_model.layers.35.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.28.mlp.down_proj": 72,
+    "model.language_model.layers.28.mlp.gate_proj": 72,
+    "model.language_model.layers.28.mlp.up_proj": 72,
+    "model.language_model.layers.28.self_attn.k_proj": 72,
+    "model.language_model.layers.28.self_attn.o_proj": 72,
+    "model.language_model.layers.28.self_attn.q_proj": 72,
+    "model.language_model.layers.28.self_attn.v_proj": 72,
+    "model.language_model.layers.29.mlp.down_proj": 72,
+    "model.language_model.layers.29.mlp.gate_proj": 72,
+    "model.language_model.layers.29.mlp.up_proj": 72,
+    "model.language_model.layers.29.self_attn.k_proj": 72,
+    "model.language_model.layers.29.self_attn.o_proj": 72,
+    "model.language_model.layers.29.self_attn.q_proj": 72,
+    "model.language_model.layers.29.self_attn.v_proj": 72,
+    "model.language_model.layers.30.mlp.down_proj": 72,
+    "model.language_model.layers.30.mlp.gate_proj": 72,
+    "model.language_model.layers.30.mlp.up_proj": 72,
+    "model.language_model.layers.30.self_attn.k_proj": 72,
+    "model.language_model.layers.30.self_attn.o_proj": 72,
+    "model.language_model.layers.30.self_attn.q_proj": 72,
+    "model.language_model.layers.30.self_attn.v_proj": 72,
+    "model.language_model.layers.31.mlp.down_proj": 72,
+    "model.language_model.layers.31.mlp.gate_proj": 72,
+    "model.language_model.layers.31.mlp.up_proj": 72,
+    "model.language_model.layers.31.self_attn.k_proj": 72,
+    "model.language_model.layers.31.self_attn.o_proj": 72,
+    "model.language_model.layers.31.self_attn.q_proj": 72,
+    "model.language_model.layers.31.self_attn.v_proj": 72,
+    "model.language_model.layers.32.mlp.down_proj": 72,
+    "model.language_model.layers.32.mlp.gate_proj": 72,
+    "model.language_model.layers.32.mlp.up_proj": 72,
+    "model.language_model.layers.32.self_attn.k_proj": 72,
+    "model.language_model.layers.32.self_attn.o_proj": 72,
+    "model.language_model.layers.32.self_attn.q_proj": 72,
+    "model.language_model.layers.32.self_attn.v_proj": 72,
+    "model.language_model.layers.33.mlp.down_proj": 72,
+    "model.language_model.layers.33.mlp.gate_proj": 72,
+    "model.language_model.layers.33.mlp.up_proj": 72,
+    "model.language_model.layers.33.self_attn.k_proj": 72,
+    "model.language_model.layers.33.self_attn.o_proj": 72,
+    "model.language_model.layers.33.self_attn.q_proj": 72,
+    "model.language_model.layers.33.self_attn.v_proj": 72,
+    "model.language_model.layers.34.mlp.down_proj": 72,
+    "model.language_model.layers.34.mlp.gate_proj": 72,
+    "model.language_model.layers.34.mlp.up_proj": 72,
+    "model.language_model.layers.34.self_attn.k_proj": 72,
+    "model.language_model.layers.34.self_attn.o_proj": 72,
+    "model.language_model.layers.34.self_attn.q_proj": 72,
+    "model.language_model.layers.34.self_attn.v_proj": 72,
+    "model.language_model.layers.35.mlp.down_proj": 72,
+    "model.language_model.layers.35.mlp.gate_proj": 72,
+    "model.language_model.layers.35.mlp.up_proj": 72,
+    "model.language_model.layers.35.self_attn.k_proj": 72,
+    "model.language_model.layers.35.self_attn.o_proj": 72,
+    "model.language_model.layers.35.self_attn.q_proj": 72,
+    "model.language_model.layers.35.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.32.mlp.gate_proj",
+    "model.language_model.layers.28.self_attn.v_proj",
+    "model.language_model.layers.30.mlp.gate_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.31.self_attn.k_proj",
+    "model.language_model.layers.33.mlp.up_proj",
+    "model.language_model.layers.30.mlp.down_proj",
+    "model.language_model.layers.31.self_attn.v_proj",
+    "model.language_model.layers.31.mlp.up_proj",
+    "model.language_model.layers.35.mlp.up_proj",
+    "model.language_model.layers.35.self_attn.q_proj",
+    "model.language_model.layers.31.mlp.down_proj",
+    "model.language_model.layers.32.self_attn.k_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.32.self_attn.q_proj",
+    "model.language_model.layers.30.self_attn.k_proj",
+    "model.language_model.layers.35.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.32.mlp.down_proj",
+    "model.language_model.layers.33.mlp.down_proj",
+    "model.language_model.layers.30.mlp.up_proj",
+    "model.language_model.layers.34.mlp.down_proj",
+    "model.language_model.layers.35.mlp.gate_proj",
+    "model.language_model.layers.29.self_attn.k_proj",
+    "model.language_model.layers.33.self_attn.q_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.35.self_attn.k_proj",
+    "model.language_model.layers.34.mlp.gate_proj",
+    "model.language_model.layers.30.self_attn.o_proj",
+    "model.language_model.layers.31.self_attn.q_proj",
+    "model.language_model.layers.33.self_attn.o_proj",
+    "model.language_model.layers.31.self_attn.o_proj",
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.33.self_attn.k_proj",
+    "model.language_model.layers.34.self_attn.o_proj",
+    "model.language_model.layers.35.mlp.down_proj",
+    "model.language_model.layers.28.mlp.gate_proj",
+    "model.language_model.layers.29.self_attn.v_proj",
+    "model.language_model.layers.32.self_attn.o_proj",
+    "model.language_model.layers.33.mlp.gate_proj",
+    "model.language_model.layers.35.self_attn.o_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.34.self_attn.q_proj",
+    "model.language_model.layers.30.self_attn.v_proj",
+    "model.language_model.layers.31.mlp.gate_proj",
+    "model.language_model.layers.34.self_attn.k_proj",
+    "model.language_model.layers.28.mlp.down_proj",
+    "model.language_model.layers.32.mlp.up_proj",
+    "model.language_model.layers.34.mlp.up_proj",
+    "model.language_model.layers.33.self_attn.v_proj",
+    "model.language_model.layers.32.self_attn.v_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.30.self_attn.q_proj",
+    "model.language_model.layers.34.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Deep8_s73/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 73,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Deep8",
+    "layers": [
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35
+    ],
+    "ranks": {
+      "28": 72,
+      "29": 72,
+      "30": 72,
+      "31": 72,
+      "32": 72,
+      "33": 72,
+      "34": 72,
+      "35": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Deep8_s73/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 4.032007694244385, "loss_scale": 4.0, "retries": 0, "seconds": 2.2, "max_memory_gb": 16.82, "ce": 0.6797246597125195, "kl": 0.0438283761832281, "context": 0.7016388622396335}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 5.783562183380127, "loss_scale": 4.0, "retries": 0, "seconds": 3.4, "max_memory_gb": 17.16, "ce": 1.7264169752597809, "kl": 0.6906487783417106, "context": 2.0717413425445557}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 3.1756043434143066, "loss_scale": 4.0, "retries": 0, "seconds": 4.6, "max_memory_gb": 17.17, "ce": 0.6097742180572823, "kl": 0.10850129910977557, "context": 0.6640248582698405}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 3.383247137069702, "loss_scale": 4.0, "retries": 0, "seconds": 5.7, "max_memory_gb": 17.17, "ce": 0.7461670730262995, "kl": 0.018099767229614372, "context": 0.7552169484551996}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 2.8684136867523193, "loss_scale": 4.0, "retries": 0, "seconds": 6.7, "max_memory_gb": 17.17, "ce": 0.3658650080978987, "kl": 0.12677907780744135, "context": 0.42925455048680305}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 2.4790596961975098, "loss_scale": 4.0, "retries": 0, "seconds": 7.7, "max_memory_gb": 17.17, "ce": 0.454464259557426, "kl": 0.07809042930603027, "context": 0.4935094714164734}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 1.1359776258468628, "loss_scale": 4.0, "retries": 0, "seconds": 8.8, "max_memory_gb": 17.17, "ce": 0.09125529236189323, "kl": 0.047093482239688456, "context": 0.11480203703285952}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.6693722009658813, "loss_scale": 4.0, "retries": 0, "seconds": 9.7, "max_memory_gb": 17.17, "ce": 0.3987525324046146, "kl": 0.514514260168653, "context": 0.6560096727334894}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 3.4810400009155273, "loss_scale": 4.0, "retries": 0, "seconds": 10.7, "max_memory_gb": 17.17, "ce": 0.5010905254166573, "kl": 0.45556964993011206, "context": 0.7288753385655582}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 1.5437238216400146, "loss_scale": 4.0, "retries": 0, "seconds": 11.8, "max_memory_gb": 17.17, "ce": 0.17057929569455155, "kl": 0.3484929918132593, "context": 0.34482580021540343}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 3.4049508571624756, "loss_scale": 4.0, "retries": 0, "seconds": 12.8, "max_memory_gb": 17.17, "ce": 0.3515164228156209, "kl": 0.07228459720499814, "context": 0.3876587129198015}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 1.3938015699386597, "loss_scale": 4.0, "retries": 0, "seconds": 13.8, "max_memory_gb": 17.17, "ce": 0.10199875673924907, "kl": 0.01780424175740336, "context": 0.11090087884804234}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 2.0758543014526367, "loss_scale": 4.0, "retries": 0, "seconds": 14.9, "max_memory_gb": 17.17, "ce": 0.1920350155705819, "kl": 0.09496072172169079, "context": 0.23951537972607184}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 0.5084232687950134, "loss_scale": 4.0, "retries": 0, "seconds": 15.9, "max_memory_gb": 17.17, "ce": 0.05480429082126648, "kl": 0.09830929344752803, "context": 0.10395893288659863}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 1.7201011180877686, "loss_scale": 4.0, "retries": 0, "seconds": 17.0, "max_memory_gb": 17.17, "ce": 0.12034604771179147, "kl": 0.13478183571714908, "context": 0.18773696187417954}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 2.282102584838867, "loss_scale": 4.0, "retries": 0, "seconds": 18.0, "max_memory_gb": 17.17, "ce": 0.7810915707523236, "kl": 0.2225360506799916, "context": 0.8923596059903502}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 1.9053561687469482, "loss_scale": 4.0, "retries": 0, "seconds": 19.0, "max_memory_gb": 17.17, "ce": 0.05448345316108316, "kl": 0.3474913834797917, "context": 0.22822914749849588}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 1.1665737628936768, "loss_scale": 4.0, "retries": 0, "seconds": 20.1, "max_memory_gb": 17.17, "ce": 0.002347836649278179, "kl": 0.2958440902584698, "context": 0.15026988449972123}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 1.3919365406036377, "loss_scale": 4.0, "retries": 0, "seconds": 21.1, "max_memory_gb": 17.17, "ce": 0.3140350406320067, "kl": 0.14251930071606012, "context": 0.38529469797504134}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 1.919246792793274, "loss_scale": 4.0, "retries": 0, "seconds": 22.1, "max_memory_gb": 17.17, "ce": 0.03979899160185596, "kl": 0.3707702329700169, "context": 0.2251841141260229}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.1500036120414734, "loss_scale": 4.0, "retries": 0, "seconds": 23.1, "max_memory_gb": 17.17, "ce": 0.005785354427644052, "kl": 0.0007848322911740979, "context": 0.006177770432259422}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 1.7179235219955444, "loss_scale": 4.0, "retries": 0, "seconds": 24.1, "max_memory_gb": 17.17, "ce": 0.15976516701630317, "kl": 0.2368797881063074, "context": 0.27820505321142264}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 1.3254204988479614, "loss_scale": 4.0, "retries": 0, "seconds": 25.1, "max_memory_gb": 17.17, "ce": 0.14706759972614236, "kl": 0.08974236852372997, "context": 0.19193879143858794}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 2.7670671939849854, "loss_scale": 4.0, "retries": 0, "seconds": 26.2, "max_memory_gb": 17.17, "ce": 0.09144656843272969, "kl": 0.2937595800613053, "context": 0.23832635849248618}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 0.49812695384025574, "loss_scale": 4.0, "retries": 0, "seconds": 27.2, "max_memory_gb": 17.17, "ce": 0.05584454696509056, "kl": 0.04908189741399838, "context": 0.08038549608318135}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 2.991490364074707, "loss_scale": 4.0, "retries": 0, "seconds": 28.3, "max_memory_gb": 17.17, "ce": 0.2647294679482002, "kl": 0.29095308274554554, "context": 0.4102060102559335}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 2.6361870765686035, "loss_scale": 4.0, "retries": 0, "seconds": 29.3, "max_memory_gb": 17.17, "ce": 0.1299037232165574, "kl": 0.17020883317172775, "context": 0.2150081471336307}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 0.049485187977552414, "loss_scale": 4.0, "retries": 0, "seconds": 30.3, "max_memory_gb": 17.17, "ce": 0.002161322205211036, "kl": 0.00032891238151933067, "context": 0.002325778390513733}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 0.26005110144615173, "loss_scale": 4.0, "retries": 0, "seconds": 31.4, "max_memory_gb": 17.17, "ce": 0.011351799766998738, "kl": 0.008757588686421514, "context": 0.015730594168417156}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.673872709274292, "loss_scale": 4.0, "retries": 0, "seconds": 32.4, "max_memory_gb": 17.17, "ce": 0.036409666179679334, "kl": 0.05665631717420183, "context": 0.06473782286047935}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 2.1549291610717773, "loss_scale": 4.0, "retries": 0, "seconds": 33.3, "max_memory_gb": 17.17, "ce": 0.008170134446118027, "kl": 0.7165377386409091, "context": 0.36643901164643466}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 4.15576696395874, "loss_scale": 4.0, "retries": 0, "seconds": 34.4, "max_memory_gb": 17.17, "ce": 0.19080939033301547, "kl": 0.3968014536476403, "context": 0.38921011658385396}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 1.126189947128296, "loss_scale": 4.0, "retries": 0, "seconds": 35.4, "max_memory_gb": 17.17, "ce": 0.02820639527635649, "kl": 0.038162208769790595, "context": 0.04728750012873206}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.5614117383956909, "loss_scale": 4.0, "retries": 0, "seconds": 36.4, "max_memory_gb": 17.17, "ce": 0.025638571682065958, "kl": 0.2978460013109725, "context": 0.17456157236301806}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.7353611588478088, "loss_scale": 4.0, "retries": 0, "seconds": 37.5, "max_memory_gb": 17.17, "ce": 0.03223461419111118, "kl": 0.08920511519710317, "context": 0.07683717412874103}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 0.8959574699401855, "loss_scale": 4.0, "retries": 0, "seconds": 38.5, "max_memory_gb": 17.17, "ce": 0.10215800530568231, "kl": 0.03543353524582926, "context": 0.11987477596267127}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 0.7327104210853577, "loss_scale": 4.0, "retries": 0, "seconds": 39.6, "max_memory_gb": 17.17, "ce": 0.010948261799057946, "kl": 0.03737581068708096, "context": 0.029636167222633958}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.35165297985076904, "loss_scale": 4.0, "retries": 0, "seconds": 40.6, "max_memory_gb": 17.17, "ce": 0.025067837224924006, "kl": 0.024584138496720698, "context": 0.03735990740824491}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.2021090984344482, "loss_scale": 4.0, "retries": 0, "seconds": 41.6, "max_memory_gb": 17.17, "ce": 0.17513403149496298, "kl": 0.02001732011558488, "context": 0.18514268775470555}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.6158775091171265, "loss_scale": 4.0, "retries": 0, "seconds": 42.6, "max_memory_gb": 17.17, "ce": 0.007308083389943931, "kl": 0.043653059809003025, "context": 0.029134613985661417}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 1.581203579902649, "loss_scale": 4.0, "retries": 0, "seconds": 43.6, "max_memory_gb": 17.17, "ce": 0.04881511820713058, "kl": 0.00399557821219787, "context": 0.05081290891394019}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 1.7906591892242432, "loss_scale": 4.0, "retries": 0, "seconds": 44.6, "max_memory_gb": 17.17, "ce": 0.12958399817580357, "kl": 0.07666294110094896, "context": 0.16791546576132532}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 4.215092658996582, "loss_scale": 4.0, "retries": 0, "seconds": 45.7, "max_memory_gb": 17.17, "ce": 0.44332390697672963, "kl": 0.1552928517339751, "context": 0.5209703310392797}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 0.18663382530212402, "loss_scale": 4.0, "retries": 0, "seconds": 46.7, "max_memory_gb": 17.17, "ce": 0.005529662710614502, "kl": 0.002081523183733225, "context": 0.006570424360688776}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 2.8400955200195312, "loss_scale": 4.0, "retries": 0, "seconds": 47.7, "max_memory_gb": 17.17, "ce": 0.0544769553525839, "kl": 0.6385753061622381, "context": 0.3737646099179983}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 1.28101646900177, "loss_scale": 4.0, "retries": 0, "seconds": 48.7, "max_memory_gb": 17.17, "ce": 0.007484978312277235, "kl": 0.7958063942134572, "context": 0.40538816284242785}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.05076250806450844, "loss_scale": 4.0, "retries": 0, "seconds": 49.7, "max_memory_gb": 17.17, "ce": 0.0028718392313749064, "kl": 0.0014642708047176711, "context": 0.0036039746773894876}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.1368301808834076, "loss_scale": 4.0, "retries": 0, "seconds": 50.8, "max_memory_gb": 17.17, "ce": 0.010851567756617442, "kl": 0.0067760132187686395, "context": 0.014239574345992878}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 1.1244195699691772, "loss_scale": 4.0, "retries": 0, "seconds": 51.8, "max_memory_gb": 17.17, "ce": 0.06885742512531579, "kl": 0.02643633907973708, "context": 0.08207559637958184}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.08164465427398682, "loss_scale": 4.0, "retries": 0, "seconds": 52.8, "max_memory_gb": 17.17, "ce": 0.003657895897049457, "kl": 0.0051857456728612306, "context": 0.006250768667086959}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.191365361213684, "loss_scale": 4.0, "retries": 0, "seconds": 53.9, "max_memory_gb": 17.17, "ce": 0.09132010361645371, "kl": 0.1338165826164186, "context": 0.15822840231703594}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 3.01678204536438, "loss_scale": 4.0, "retries": 0, "seconds": 54.9, "max_memory_gb": 17.17, "ce": 0.37840992922428995, "kl": 0.3803166052530287, "context": 0.5685682336916216}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 3.5693399906158447, "loss_scale": 4.0, "retries": 0, "seconds": 55.8, "max_memory_gb": 17.17, "ce": 0.6015139933169849, "kl": 0.41397527484514285, "context": 0.8085016157128848}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 1.2623422145843506, "loss_scale": 4.0, "retries": 0, "seconds": 56.9, "max_memory_gb": 17.17, "ce": 0.3513382289092988, "kl": 0.07210659431439126, "context": 0.38739151670597494}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 2.7980635166168213, "loss_scale": 4.0, "retries": 0, "seconds": 57.9, "max_memory_gb": 17.17, "ce": 0.14985343755688518, "kl": 0.008009930024854839, "context": 0.15385840251110494}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 1.7308049201965332, "loss_scale": 4.0, "retries": 0, "seconds": 58.9, "max_memory_gb": 17.17, "ce": 0.0019403475453145802, "kl": 0.17619746051786933, "context": 0.09003907936858013}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 2.096055030822754, "loss_scale": 4.0, "retries": 0, "seconds": 60.0, "max_memory_gb": 17.17, "ce": 0.010325112256396096, "kl": 0.4475945309004601, "context": 0.234122379188193}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.9929479956626892, "loss_scale": 4.0, "retries": 0, "seconds": 60.9, "max_memory_gb": 17.17, "ce": 0.0021519916772376746, "kl": 0.0950060307077365, "context": 0.04965500542311929}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.28454849123954773, "loss_scale": 4.0, "retries": 0, "seconds": 62.0, "max_memory_gb": 17.17, "ce": 0.008366316033061594, "kl": 0.0032855163372005336, "context": 0.010009074314439204}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 0.21614119410514832, "loss_scale": 4.0, "retries": 0, "seconds": 63.0, "max_memory_gb": 17.17, "ce": 0.012862431292887777, "kl": 0.012402385531459004, "context": 0.01906362408772111}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 1.9976236820220947, "loss_scale": 4.0, "retries": 0, "seconds": 64.0, "max_memory_gb": 17.17, "ce": 0.17313899914734066, "kl": 0.22828934862627648, "context": 0.2872836741153151}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 1.7251005172729492, "loss_scale": 4.0, "retries": 0, "seconds": 65.1, "max_memory_gb": 17.17, "ce": 0.39216809908975847, "kl": 0.031200174380501267, "context": 0.40776817343430594}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 0.23958176374435425, "loss_scale": 4.0, "retries": 0, "seconds": 66.1, "max_memory_gb": 17.17, "ce": 0.02824014596080815, "kl": 0.0138411808084129, "context": 0.035160735438694246}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 3.8207643032073975, "loss_scale": 4.0, "retries": 0, "seconds": 67.1, "max_memory_gb": 17.17, "ce": 0.42285745666958974, "kl": 0.011287840796285309, "context": 0.4285013658227399}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 1.0578058958053589, "loss_scale": 4.0, "retries": 0, "seconds": 68.2, "max_memory_gb": 17.17, "ce": 0.05214453669032082, "kl": 0.014487280452158302, "context": 0.05938817746937275}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 0.12342632561922073, "loss_scale": 4.0, "retries": 0, "seconds": 69.1, "max_memory_gb": 17.17, "ce": 0.004258734290488064, "kl": 0.0031623230897821486, "context": 0.005839895864482969}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.8736779093742371, "loss_scale": 4.0, "retries": 0, "seconds": 70.1, "max_memory_gb": 17.17, "ce": 0.2799174004467204, "kl": 0.05261199356755242, "context": 0.30622341320849955}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 1.1636828184127808, "loss_scale": 4.0, "retries": 0, "seconds": 71.2, "max_memory_gb": 17.17, "ce": 0.048779021151858615, "kl": 0.008290437661344185, "context": 0.05292424085200764}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 0.5804976224899292, "loss_scale": 4.0, "retries": 0, "seconds": 72.2, "max_memory_gb": 17.17, "ce": 0.04926839972904418, "kl": 0.12495812793349614, "context": 0.11174745995958801}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.2432561218738556, "loss_scale": 4.0, "retries": 0, "seconds": 73.3, "max_memory_gb": 17.17, "ce": 0.011045366176404059, "kl": 0.07027242382991972, "context": 0.0461815771413967}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 1.6426793336868286, "loss_scale": 4.0, "retries": 0, "seconds": 74.3, "max_memory_gb": 17.17, "ce": 0.0022592152818106115, "kl": 0.17219388015018922, "context": 0.08835615569842048}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.11708974093198776, "loss_scale": 4.0, "retries": 0, "seconds": 75.3, "max_memory_gb": 17.17, "ce": 0.008917873674363364, "kl": 0.014278097254646127, "context": 0.01605692190787522}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 0.675159752368927, "loss_scale": 4.0, "retries": 0, "seconds": 76.4, "max_memory_gb": 17.17, "ce": 0.018520939636800904, "kl": 0.19395656472624978, "context": 0.11549922199628782}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.0876054763793945, "loss_scale": 4.0, "retries": 0, "seconds": 77.3, "max_memory_gb": 17.17, "ce": 0.05296102201100439, "kl": 0.047866615204839036, "context": 0.0768943295115605}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 0.24891580641269684, "loss_scale": 4.0, "retries": 0, "seconds": 78.3, "max_memory_gb": 17.17, "ce": 0.009787061248061946, "kl": 0.01775225322762708, "context": 0.018663188093341887}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 0.9323069453239441, "loss_scale": 4.0, "retries": 0, "seconds": 79.4, "max_memory_gb": 17.17, "ce": 0.3859800465288572, "kl": 0.04579554616634596, "context": 0.40887781589117367}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.2712996006011963, "loss_scale": 4.0, "retries": 0, "seconds": 80.4, "max_memory_gb": 17.17, "ce": 0.0015600175829604268, "kl": 0.024004426086321473, "context": 0.01356223039329052}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 1.2446175813674927, "loss_scale": 4.0, "retries": 0, "seconds": 81.4, "max_memory_gb": 17.17, "ce": 0.0026389116246718913, "kl": 0.7234501839448058, "context": 0.36436399834929034}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.1561068892478943, "loss_scale": 4.0, "retries": 0, "seconds": 82.5, "max_memory_gb": 17.17, "ce": 0.011692562606185675, "kl": 0.008396799632464536, "context": 0.01589096209499985}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 1.0383491516113281, "loss_scale": 4.0, "retries": 0, "seconds": 83.5, "max_memory_gb": 17.17, "ce": 0.2737160621691146, "kl": 0.03576084706992333, "context": 0.29159647072083317}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 2.8752553462982178, "loss_scale": 4.0, "retries": 0, "seconds": 84.6, "max_memory_gb": 17.17, "ce": 0.22086436792596942, "kl": 0.04989777015725849, "context": 0.24581325890903827}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.3004063069820404, "loss_scale": 4.0, "retries": 0, "seconds": 85.6, "max_memory_gb": 17.17, "ce": 0.013719018723350018, "kl": 0.006011663055687677, "context": 0.016724850807804614}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 2.45418643951416, "loss_scale": 4.0, "retries": 0, "seconds": 86.6, "max_memory_gb": 17.17, "ce": 0.0011498305066197645, "kl": 0.41520564439542795, "context": 0.2087526466784766}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 2.306347608566284, "loss_scale": 4.0, "retries": 0, "seconds": 87.7, "max_memory_gb": 17.17, "ce": 0.00534782432805514, "kl": 0.43738872523317696, "context": 0.22404219325107988}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.27115997672080994, "loss_scale": 4.0, "retries": 0, "seconds": 88.6, "max_memory_gb": 17.17, "ce": 0.014828865067102015, "kl": 0.015105128162758774, "context": 0.02238142959686229}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.7064262628555298, "loss_scale": 4.0, "retries": 0, "seconds": 89.6, "max_memory_gb": 17.17, "ce": 0.005868612737685908, "kl": 0.39309596076782327, "context": 0.2024165897310013}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.07430069148540497, "loss_scale": 4.0, "retries": 0, "seconds": 90.7, "max_memory_gb": 17.17, "ce": 0.004765952195157297, "kl": 0.0026626978469721507, "context": 0.006097300967667252}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.4741643965244293, "loss_scale": 4.0, "retries": 0, "seconds": 91.7, "max_memory_gb": 17.17, "ce": 0.02841635752702132, "kl": 0.04777639862504657, "context": 0.052304554970760364}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.8067178130149841, "loss_scale": 4.0, "retries": 0, "seconds": 92.7, "max_memory_gb": 17.17, "ce": 0.08084095394588076, "kl": 0.17104944681523193, "context": 0.16636567175737582}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.41732853651046753, "loss_scale": 4.0, "retries": 0, "seconds": 93.8, "max_memory_gb": 17.17, "ce": 0.04827791510615498, "kl": 0.05267232009828149, "context": 0.07461407521623187}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 0.6030309796333313, "loss_scale": 4.0, "retries": 0, "seconds": 94.7, "max_memory_gb": 17.17, "ce": 0.07720415928633884, "kl": 0.07844582560937852, "context": 0.11642707174178213}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 0.7796145081520081, "loss_scale": 4.0, "retries": 0, "seconds": 95.8, "max_memory_gb": 17.17, "ce": 0.035799885787128005, "kl": 0.15109330700943246, "context": 0.11134654117631726}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 1.241249680519104, "loss_scale": 4.0, "retries": 0, "seconds": 96.8, "max_memory_gb": 17.17, "ce": 0.21570116307702847, "kl": 0.3296090420335531, "context": 0.3805056873243302}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 0.05933153256773949, "loss_scale": 4.0, "retries": 0, "seconds": 97.8, "max_memory_gb": 17.17, "ce": 0.004357696496299468, "kl": 0.002687567410248448, "context": 0.005701480302377604}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 1.7564656734466553, "loss_scale": 4.0, "retries": 0, "seconds": 98.9, "max_memory_gb": 17.17, "ce": 0.17380010413762648, "kl": 0.9368478331525694, "context": 0.6422240181127563}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 1.200290322303772, "loss_scale": 4.0, "retries": 0, "seconds": 99.9, "max_memory_gb": 17.17, "ce": 0.04494663016521372, "kl": 0.09994048997759819, "context": 0.09491687244735658}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.5392159223556519, "loss_scale": 4.0, "retries": 0, "seconds": 100.9, "max_memory_gb": 17.17, "ce": 0.014910284575307742, "kl": 0.03713451532894396, "context": 0.03347754158312455}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.13016141951084137, "loss_scale": 4.0, "retries": 0, "seconds": 102.0, "max_memory_gb": 17.17, "ce": 0.010535652283579111, "kl": 0.005737312039855169, "context": 0.01340430835261941}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.2833978831768036, "loss_scale": 4.0, "retries": 0, "seconds": 103.0, "max_memory_gb": 17.17, "ce": 0.03822777574532665, "kl": 0.06046390900610277, "context": 0.06845973030431196}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.39993569254875183, "loss_scale": 4.0, "retries": 0, "seconds": 103.9, "max_memory_gb": 17.17, "ce": 0.05016736072138883, "kl": 0.06771189413848333, "context": 0.08402330777607858}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 1.848693609237671, "loss_scale": 4.0, "retries": 0, "seconds": 105.0, "max_memory_gb": 17.17, "ce": 0.32966657308861613, "kl": 0.0992694670567289, "context": 0.3793013205868192}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.38049671053886414, "loss_scale": 4.0, "retries": 0, "seconds": 106.0, "max_memory_gb": 17.17, "ce": 0.01906500296900049, "kl": 0.03201814749627374, "context": 0.03507407708093524}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.0922410860657692, "loss_scale": 4.0, "retries": 0, "seconds": 107.1, "max_memory_gb": 17.17, "ce": 0.0077033768757246435, "kl": 0.00448610292187368, "context": 0.009946427890099585}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 4.129609107971191, "loss_scale": 4.0, "retries": 0, "seconds": 108.1, "max_memory_gb": 17.17, "ce": 0.3362821593909757, "kl": 0.295363093377091, "context": 0.4839636944234371}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 0.6048001050949097, "loss_scale": 4.0, "retries": 0, "seconds": 109.1, "max_memory_gb": 17.17, "ce": 0.11792916967533529, "kl": 0.03492555979028111, "context": 0.1353919489774853}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.8190692663192749, "loss_scale": 4.0, "retries": 0, "seconds": 110.2, "max_memory_gb": 17.17, "ce": 0.03777283325325698, "kl": 0.15268607903271914, "context": 0.11411587311886251}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.180077463388443, "loss_scale": 4.0, "retries": 0, "seconds": 111.2, "max_memory_gb": 17.17, "ce": 0.004114289855351672, "kl": 0.01738534765172517, "context": 0.012806963408365846}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 0.25962018966674805, "loss_scale": 4.0, "retries": 0, "seconds": 112.2, "max_memory_gb": 17.17, "ce": 0.020637889741919935, "kl": 0.008690844217198901, "context": 0.024983312468975782}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.17715470492839813, "loss_scale": 4.0, "retries": 0, "seconds": 113.3, "max_memory_gb": 17.17, "ce": 0.0037316386806196533, "kl": 0.01606105369864963, "context": 0.011762165377149358}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.11712247878313065, "loss_scale": 4.0, "retries": 0, "seconds": 114.2, "max_memory_gb": 17.17, "ce": 0.008764029073063284, "kl": 0.002788148103718413, "context": 0.010158103366848081}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.8342969417572021, "loss_scale": 4.0, "retries": 0, "seconds": 115.2, "max_memory_gb": 17.17, "ce": 0.19377320239436813, "kl": 0.001071026745648851, "context": 0.19430871194344945}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.23988835513591766, "loss_scale": 4.0, "retries": 0, "seconds": 116.3, "max_memory_gb": 17.17, "ce": 0.01119928405387327, "kl": 0.00981819130902295, "context": 0.01610837981570512}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 1.768725037574768, "loss_scale": 4.0, "retries": 0, "seconds": 117.3, "max_memory_gb": 17.17, "ce": 0.527097261408926, "kl": 0.04910862418910256, "context": 0.5516515846247785}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.7893430590629578, "loss_scale": 4.0, "retries": 0, "seconds": 118.4, "max_memory_gb": 17.17, "ce": 0.21845766436308622, "kl": 0.10972363036125898, "context": 0.27331947442144156}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 2.4548680782318115, "loss_scale": 4.0, "retries": 0, "seconds": 119.4, "max_memory_gb": 17.17, "ce": 0.10789192689117044, "kl": 0.06190183905619051, "context": 0.1388428467034828}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 1.5827323198318481, "loss_scale": 4.0, "retries": 0, "seconds": 120.4, "max_memory_gb": 17.17, "ce": 0.3597470640670508, "kl": 0.021876350743696094, "context": 0.3706852470058948}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.16418641805648804, "loss_scale": 4.0, "retries": 0, "seconds": 121.5, "max_memory_gb": 17.17, "ce": 0.009772353179869242, "kl": 0.018305232737475308, "context": 0.01892496956861578}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 0.8646790385246277, "loss_scale": 4.0, "retries": 0, "seconds": 122.5, "max_memory_gb": 17.17, "ce": 0.07771265157498419, "kl": 0.04631816613255069, "context": 0.10087173851206899}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.08572100847959518, "loss_scale": 4.0, "retries": 0, "seconds": 123.5, "max_memory_gb": 17.17, "ce": 0.009499331092229113, "kl": 0.0004882854045717977, "context": 0.009743473696289584}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 2.0778329372406006, "loss_scale": 4.0, "retries": 0, "seconds": 124.6, "max_memory_gb": 17.17, "ce": 0.2379631840158254, "kl": 0.1365643609315157, "context": 0.3062453716993332}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.9593974947929382, "loss_scale": 4.0, "retries": 0, "seconds": 125.6, "max_memory_gb": 17.17, "ce": 0.030800518346950412, "kl": 0.5160025017321459, "context": 0.2888017753139138}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 0.8819509148597717, "loss_scale": 4.0, "retries": 0, "seconds": 126.6, "max_memory_gb": 17.17, "ce": 0.17105657306092326, "kl": 0.011477154825115576, "context": 0.1767951447982341}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 1.2876622676849365, "loss_scale": 4.0, "retries": 0, "seconds": 127.7, "max_memory_gb": 17.17, "ce": 0.3708801147004124, "kl": 0.044012651676894166, "context": 0.39288644105545245}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 0.15604561567306519, "loss_scale": 4.0, "retries": 0, "seconds": 128.7, "max_memory_gb": 17.17, "ce": 0.002135812203050591, "kl": 0.017570545183843933, "context": 0.010921084962319583}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 1.4046486616134644, "loss_scale": 4.0, "retries": 0, "seconds": 129.8, "max_memory_gb": 17.17, "ce": 0.04854234866797924, "kl": 1.0727308802306652, "context": 0.5849077925086021}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 0.29337021708488464, "loss_scale": 4.0, "retries": 0, "seconds": 130.9, "max_memory_gb": 17.17, "ce": 0.03298717987490818, "kl": 0.03163624324952252, "context": 0.04880530200898647}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.0395963229238987, "loss_scale": 4.0, "retries": 0, "seconds": 131.9, "max_memory_gb": 17.17, "ce": 0.0014838135684840381, "kl": 0.0030464035626209807, "context": 0.0030070153588894755}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.19924405217170715, "loss_scale": 4.0, "retries": 0, "seconds": 133.0, "max_memory_gb": 17.17, "ce": 0.016363823844585568, "kl": 0.014410491567105055, "context": 0.02356906933709979}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 0.46190717816352844, "loss_scale": 4.0, "retries": 0, "seconds": 134.0, "max_memory_gb": 17.17, "ce": 0.050257908704224974, "kl": 0.012790942912033643, "context": 0.056653380393981934}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 1.2393771409988403, "loss_scale": 4.0, "retries": 0, "seconds": 134.9, "max_memory_gb": 17.17, "ce": 0.08985339890932664, "kl": 0.11879819142632186, "context": 0.14925249456427991}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 0.7575793862342834, "loss_scale": 4.0, "retries": 0, "seconds": 136.1, "max_memory_gb": 17.17, "ce": 0.09632438301923685, "kl": 0.2134148169116088, "context": 0.20303179149050266}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.42120710015296936, "loss_scale": 4.0, "retries": 0, "seconds": 137.0, "max_memory_gb": 17.17, "ce": 0.017289109411649406, "kl": 0.01260900218039751, "context": 0.02359361050184816}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.7787611484527588, "loss_scale": 4.0, "retries": 0, "seconds": 138.1, "max_memory_gb": 17.17, "ce": 0.07690849788195919, "kl": 0.06091404838662129, "context": 0.10736552369780838}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.0879928320646286, "loss_scale": 4.0, "retries": 0, "seconds": 139.1, "max_memory_gb": 17.17, "ce": 0.007857171935029328, "kl": 0.002177803944960033, "context": 0.008946073881816119}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 1.1368129253387451, "loss_scale": 4.0, "retries": 0, "seconds": 140.1, "max_memory_gb": 17.17, "ce": 0.08451696614793036, "kl": 0.003774205591980717, "context": 0.0864040661836043}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 2.343569755554199, "loss_scale": 4.0, "retries": 0, "seconds": 141.1, "max_memory_gb": 17.17, "ce": 0.6552342411596328, "kl": 0.23091133966227062, "context": 0.7706899184267968}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 1.4557684659957886, "loss_scale": 4.0, "retries": 0, "seconds": 142.0, "max_memory_gb": 17.17, "ce": 0.18451454347814433, "kl": 0.004123895887914841, "context": 0.18657648417865857}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.08832648396492004, "loss_scale": 4.0, "retries": 0, "seconds": 142.9, "max_memory_gb": 17.17, "ce": 0.009960260358639061, "kl": 0.0025415133500246156, "context": 0.011231016687816009}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 0.09733855724334717, "loss_scale": 4.0, "retries": 0, "seconds": 143.8, "max_memory_gb": 17.17, "ce": 0.009485007205512375, "kl": 0.006525638164021075, "context": 0.012747826520353556}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.4341698884963989, "loss_scale": 4.0, "retries": 0, "seconds": 144.7, "max_memory_gb": 17.17, "ce": 0.0323024504177738, "kl": 0.00568498418647323, "context": 0.035144943452905864}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.07241705060005188, "loss_scale": 4.0, "retries": 0, "seconds": 145.6, "max_memory_gb": 17.17, "ce": 0.007348388287937269, "kl": 0.006344001245452091, "context": 0.010520388896111399}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 0.4143351912498474, "loss_scale": 4.0, "retries": 0, "seconds": 146.6, "max_memory_gb": 17.17, "ce": 0.11140637809876353, "kl": 0.08563764271093532, "context": 0.1542252004146576}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 1.6937330961227417, "loss_scale": 4.0, "retries": 0, "seconds": 147.5, "max_memory_gb": 17.17, "ce": 0.2195807023672387, "kl": 0.4515965790166092, "context": 0.445378992066253}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.29143568873405457, "loss_scale": 4.0, "retries": 0, "seconds": 148.4, "max_memory_gb": 17.17, "ce": 0.006660997663857415, "kl": 0.026339229067161796, "context": 0.019830611941870302}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 2.4548003673553467, "loss_scale": 4.0, "retries": 0, "seconds": 149.4, "max_memory_gb": 17.17, "ce": 0.13762563522323035, "kl": 0.20612414917559363, "context": 0.24068771547172219}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.9930684566497803, "loss_scale": 4.0, "retries": 0, "seconds": 150.3, "max_memory_gb": 17.17, "ce": 0.18340046022785828, "kl": 0.008205548510886729, "context": 0.18750323518179357}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 0.7876977324485779, "loss_scale": 4.0, "retries": 0, "seconds": 151.3, "max_memory_gb": 17.17, "ce": 0.07094976049847901, "kl": 0.005146142386365682, "context": 0.07352283125510439}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.8811540007591248, "loss_scale": 4.0, "retries": 0, "seconds": 152.2, "max_memory_gb": 17.17, "ce": 0.0016881404153537005, "kl": 0.11714608373586088, "context": 0.06026118333102204}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 1.90347421169281, "loss_scale": 4.0, "retries": 0, "seconds": 153.0, "max_memory_gb": 17.17, "ce": 0.6783963609486818, "kl": 0.12371817227540305, "context": 0.7402554373256862}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 2.0315101146698, "loss_scale": 4.0, "retries": 0, "seconds": 154.0, "max_memory_gb": 17.17, "ce": 0.008458098222035915, "kl": 0.351253501517931, "context": 0.18408484989777207}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.48532599210739136, "loss_scale": 4.0, "retries": 0, "seconds": 154.9, "max_memory_gb": 17.17, "ce": 0.13920764630893245, "kl": 0.04626970293611521, "context": 0.16234249470289797}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 1.1441987752914429, "loss_scale": 4.0, "retries": 0, "seconds": 155.8, "max_memory_gb": 17.17, "ce": 0.1310053406341467, "kl": 0.6081314778421074, "context": 0.4350710795260966}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.06759079545736313, "loss_scale": 4.0, "retries": 0, "seconds": 156.8, "max_memory_gb": 17.17, "ce": 0.0061845966265536845, "kl": 0.0024067614708656038, "context": 0.0073879773408407345}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.9186450242996216, "loss_scale": 4.0, "retries": 0, "seconds": 157.6, "max_memory_gb": 17.17, "ce": 0.12501194421201944, "kl": 0.03909621988350409, "context": 0.14456005251850002}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.3751033842563629, "loss_scale": 4.0, "retries": 0, "seconds": 158.5, "max_memory_gb": 17.17, "ce": 0.037429382573463954, "kl": 0.031036621508064854, "context": 0.05294769536703825}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.247844398021698, "loss_scale": 4.0, "retries": 0, "seconds": 159.5, "max_memory_gb": 17.17, "ce": 0.028400678653270006, "kl": 0.007941818941617385, "context": 0.032371588808018714}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 2.120271921157837, "loss_scale": 4.0, "retries": 0, "seconds": 160.4, "max_memory_gb": 17.17, "ce": 0.258205725956941, "kl": 0.25143061274138745, "context": 0.3839210230798926}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 0.2608060836791992, "loss_scale": 4.0, "retries": 0, "seconds": 161.4, "max_memory_gb": 17.17, "ce": 0.005971074657281861, "kl": 0.02116918702085968, "context": 0.01655566771660233}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.061807651072740555, "loss_scale": 4.0, "retries": 0, "seconds": 162.3, "max_memory_gb": 17.17, "ce": 0.00477031077753054, "kl": 0.0013087143622669828, "context": 0.005424668008345179}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.7067413330078125, "loss_scale": 4.0, "retries": 0, "seconds": 163.2, "max_memory_gb": 17.17, "ce": 0.017961103498237208, "kl": 0.25759140424634097, "context": 0.14675681150401942}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 0.9697496294975281, "loss_scale": 4.0, "retries": 0, "seconds": 164.1, "max_memory_gb": 17.17, "ce": 0.05339793967141304, "kl": 0.11061043893096212, "context": 0.10870315726788249}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 0.4785604178905487, "loss_scale": 4.0, "retries": 0, "seconds": 165.0, "max_memory_gb": 17.17, "ce": 0.01759247900918126, "kl": 0.06509774894220755, "context": 0.050141353625804186}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 0.3586170971393585, "loss_scale": 4.0, "retries": 0, "seconds": 165.9, "max_memory_gb": 17.17, "ce": 0.040081220387946814, "kl": 0.048714460848714225, "context": 0.06443845288595185}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.9901804327964783, "loss_scale": 4.0, "retries": 0, "seconds": 166.9, "max_memory_gb": 17.17, "ce": 0.062062623677775264, "kl": 0.053393418565065076, "context": 0.08875933242961764}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.1091531440615654, "loss_scale": 4.0, "retries": 0, "seconds": 167.8, "max_memory_gb": 17.17, "ce": 0.0059807575307786465, "kl": 0.00368586863623932, "context": 0.007823691936209798}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.12318968772888184, "loss_scale": 4.0, "retries": 0, "seconds": 168.7, "max_memory_gb": 17.17, "ce": 0.011031954665668309, "kl": 0.0006557323777087731, "context": 0.011359820811776444}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.9289318919181824, "loss_scale": 4.0, "retries": 0, "seconds": 169.6, "max_memory_gb": 17.17, "ce": 0.29533912270562723, "kl": 0.010857402448891662, "context": 0.3007678385474719}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 1.82240629196167, "loss_scale": 4.0, "retries": 0, "seconds": 170.5, "max_memory_gb": 17.17, "ce": 0.21673183818347752, "kl": 0.051400696276687086, "context": 0.24243219231721014}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 0.06844368577003479, "loss_scale": 4.0, "retries": 0, "seconds": 171.5, "max_memory_gb": 17.17, "ce": 0.005828931665746495, "kl": 0.0012049825854774099, "context": 0.006431423011235893}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.5702540874481201, "loss_scale": 4.0, "retries": 0, "seconds": 172.4, "max_memory_gb": 17.17, "ce": 0.1448374113533646, "kl": 0.004452704233699478, "context": 0.14706377044785768}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.5093551874160767, "loss_scale": 4.0, "retries": 0, "seconds": 173.3, "max_memory_gb": 17.17, "ce": 0.0783587716287002, "kl": 0.08455149072688073, "context": 0.12063451705034822}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 1.1026148796081543, "loss_scale": 4.0, "retries": 0, "seconds": 174.2, "max_memory_gb": 17.17, "ce": 0.1996395201422274, "kl": 0.06246378039941192, "context": 0.23087140126153827}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.32223474979400635, "loss_scale": 4.0, "retries": 0, "seconds": 175.1, "max_memory_gb": 17.17, "ce": 0.05358947080094367, "kl": 0.012771614040389068, "context": 0.059975277370540425}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.8231096267700195, "loss_scale": 4.0, "retries": 0, "seconds": 176.0, "max_memory_gb": 17.17, "ce": 0.12024259089957923, "kl": 0.030041795487704803, "context": 0.13526348324376158}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 1.1702696084976196, "loss_scale": 4.0, "retries": 0, "seconds": 177.0, "max_memory_gb": 17.17, "ce": 0.1702803794760257, "kl": 0.3685180348693393, "context": 0.35453939717262983}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.6362076997756958, "loss_scale": 4.0, "retries": 0, "seconds": 177.9, "max_memory_gb": 17.17, "ce": 0.12588845897698775, "kl": 0.024352059175726026, "context": 0.13806448626564816}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.6941398978233337, "loss_scale": 4.0, "retries": 0, "seconds": 178.8, "max_memory_gb": 17.17, "ce": 0.10541088308673352, "kl": 0.03664789960748749, "context": 0.12373483006376773}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.13104397058486938, "loss_scale": 4.0, "retries": 0, "seconds": 179.8, "max_memory_gb": 17.17, "ce": 0.008612613819423132, "kl": 0.000989800547813502, "context": 0.009107514095376246}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 0.46230006217956543, "loss_scale": 4.0, "retries": 0, "seconds": 180.6, "max_memory_gb": 17.17, "ce": 0.100896645031753, "kl": 0.010235072964860592, "context": 0.10601417772704735}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.14150922000408173, "loss_scale": 4.0, "retries": 0, "seconds": 181.6, "max_memory_gb": 17.17, "ce": 0.007751210418064147, "kl": 0.02155563222731871, "context": 0.01852902688551694}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 0.740304172039032, "loss_scale": 4.0, "retries": 0, "seconds": 182.5, "max_memory_gb": 17.17, "ce": 0.15700226975604892, "kl": 0.03699524619150907, "context": 0.17549989488907158}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 1.077604055404663, "loss_scale": 4.0, "retries": 0, "seconds": 183.4, "max_memory_gb": 17.17, "ce": 0.0017866302805487067, "kl": 0.11507585873187054, "context": 0.05932455789297819}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.8223061561584473, "loss_scale": 4.0, "retries": 0, "seconds": 184.4, "max_memory_gb": 17.17, "ce": 0.007682784926146269, "kl": 0.12201380947226426, "context": 0.06868969229981303}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.834884524345398, "loss_scale": 4.0, "retries": 0, "seconds": 185.3, "max_memory_gb": 17.17, "ce": 0.26640599133679643, "kl": 0.05027401790721342, "context": 0.29154298931825906}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.8140994906425476, "loss_scale": 4.0, "retries": 0, "seconds": 186.2, "max_memory_gb": 17.17, "ce": 0.07295544832595624, "kl": 0.022798547528509516, "context": 0.0843547219701577}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 0.5922626256942749, "loss_scale": 4.0, "retries": 0, "seconds": 187.1, "max_memory_gb": 17.17, "ce": 0.03615815710509196, "kl": 0.10574969545268686, "context": 0.08903300482779741}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 2.4354357719421387, "loss_scale": 4.0, "retries": 0, "seconds": 188.0, "max_memory_gb": 17.17, "ce": 0.1703815836517606, "kl": 0.21824719094729517, "context": 0.2795051828434225}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 0.7700040340423584, "loss_scale": 4.0, "retries": 0, "seconds": 188.9, "max_memory_gb": 17.17, "ce": 0.06627459582523443, "kl": 0.07896593168698018, "context": 0.10575755735044368}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 0.706867516040802, "loss_scale": 4.0, "retries": 0, "seconds": 189.9, "max_memory_gb": 17.17, "ce": 0.008894435944966972, "kl": 0.15781213895752444, "context": 0.08780050801578909}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 2.300659656524658, "loss_scale": 4.0, "retries": 0, "seconds": 190.8, "max_memory_gb": 17.17, "ce": 0.2214427958097076, "kl": 0.248513841201202, "context": 0.34569972366443835}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.07423846423625946, "loss_scale": 4.0, "retries": 0, "seconds": 191.8, "max_memory_gb": 17.17, "ce": 0.0057122097932733595, "kl": 0.0008365992835024372, "context": 0.006130509544163942}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 0.5876277089118958, "loss_scale": 4.0, "retries": 0, "seconds": 192.7, "max_memory_gb": 17.17, "ce": 0.01103244925616309, "kl": 0.057241172282374464, "context": 0.03965303592849523}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.13116709887981415, "loss_scale": 4.0, "retries": 0, "seconds": 193.6, "max_memory_gb": 17.17, "ce": 0.010541612340603024, "kl": 0.018488360939954873, "context": 0.019785793672781438}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.9449759125709534, "loss_scale": 4.0, "retries": 0, "seconds": 194.5, "max_memory_gb": 17.17, "ce": 0.04896495374850929, "kl": 0.14057169695661287, "context": 0.11925080220680684}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.3445792496204376, "loss_scale": 4.0, "retries": 0, "seconds": 195.4, "max_memory_gb": 17.17, "ce": 0.037114074802957475, "kl": 0.08859413911704905, "context": 0.0814111481886357}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.07832254469394684, "loss_scale": 4.0, "retries": 0, "seconds": 196.3, "max_memory_gb": 17.17, "ce": 0.004546950047370046, "kl": 0.002456668549712049, "context": 0.005775284196715802}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.01832924783229828, "loss_scale": 4.0, "retries": 0, "seconds": 197.3, "max_memory_gb": 17.17, "ce": 0.00139901545480825, "kl": 0.00027158848752151243, "context": 0.0015348097076639533}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.8649104833602905, "loss_scale": 4.0, "retries": 0, "seconds": 198.2, "max_memory_gb": 17.17, "ce": 0.06910119188250974, "kl": 0.39183285478065955, "context": 0.26501762331463397}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 0.02623588964343071, "loss_scale": 4.0, "retries": 0, "seconds": 199.0, "max_memory_gb": 17.17, "ce": 0.002233550672826823, "kl": 0.0006859170766801981, "context": 0.002576509265054483}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.0565897226333618, "loss_scale": 4.0, "retries": 0, "seconds": 200.0, "max_memory_gb": 17.17, "ce": 0.021004752605222166, "kl": 0.722228231607005, "context": 0.3821188719011843}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.6637985110282898, "loss_scale": 4.0, "retries": 0, "seconds": 200.9, "max_memory_gb": 17.17, "ce": 0.05599350012198556, "kl": 0.008486453807563521, "context": 0.06023672746960074}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.21285909414291382, "loss_scale": 4.0, "retries": 0, "seconds": 201.9, "max_memory_gb": 17.17, "ce": 0.029045753806713037, "kl": 0.00411562262706866, "context": 0.031103565896046348}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.5813673138618469, "loss_scale": 4.0, "retries": 0, "seconds": 202.8, "max_memory_gb": 17.17, "ce": 0.06724611175013706, "kl": 0.042030470562167466, "context": 0.08826134447008371}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.054298240691423416, "loss_scale": 4.0, "retries": 0, "seconds": 203.7, "max_memory_gb": 17.17, "ce": 0.0046099206956569105, "kl": 0.0008883296832209453, "context": 0.00505408551543951}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 1.1900421380996704, "loss_scale": 4.0, "retries": 0, "seconds": 204.6, "max_memory_gb": 17.17, "ce": 0.29263457425986417, "kl": 0.00826062860141974, "context": 0.2967648758785799}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 2.225412607192993, "loss_scale": 4.0, "retries": 0, "seconds": 205.5, "max_memory_gb": 17.17, "ce": 0.28037760668667033, "kl": 0.5396691010137147, "context": 0.5502121523604728}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.2396177053451538, "loss_scale": 4.0, "retries": 0, "seconds": 206.4, "max_memory_gb": 17.17, "ce": 0.029119407481630333, "kl": 0.013220903890527325, "context": 0.03572985848586541}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.14492079615592957, "loss_scale": 4.0, "retries": 0, "seconds": 207.4, "max_memory_gb": 17.17, "ce": 0.0145880937052425, "kl": 0.000792197126429528, "context": 0.014984192472184077}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.08112093061208725, "loss_scale": 4.0, "retries": 0, "seconds": 208.3, "max_memory_gb": 17.17, "ce": 0.005773472657892853, "kl": 0.004494450578931719, "context": 0.008020697860047221}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 0.3852442502975464, "loss_scale": 4.0, "retries": 0, "seconds": 209.2, "max_memory_gb": 17.17, "ce": 0.03108427744882647, "kl": 0.001805649385801189, "context": 0.03198710276046768}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 2.048196792602539, "loss_scale": 4.0, "retries": 0, "seconds": 210.2, "max_memory_gb": 17.17, "ce": 0.602245313799358, "kl": 0.21604626381304115, "context": 0.7102684346027672}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 2.2035624980926514, "loss_scale": 4.0, "retries": 0, "seconds": 211.0, "max_memory_gb": 17.17, "ce": 0.06573927029967308, "kl": 0.39662013690758613, "context": 0.26404933966114186}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 1.0155473947525024, "loss_scale": 4.0, "retries": 0, "seconds": 212.0, "max_memory_gb": 17.17, "ce": 0.03936526668258011, "kl": 0.02843762093107216, "context": 0.05358407902531326}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.3568008244037628, "loss_scale": 4.0, "retries": 0, "seconds": 212.9, "max_memory_gb": 17.17, "ce": 0.0650838022120297, "kl": 0.030912979564163834, "context": 0.08054028986953199}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.22600305080413818, "loss_scale": 4.0, "retries": 0, "seconds": 213.8, "max_memory_gb": 17.17, "ce": 0.02545850188471377, "kl": 0.017910323571413755, "context": 0.03441366401966661}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.975039005279541, "loss_scale": 4.0, "retries": 0, "seconds": 214.8, "max_memory_gb": 17.17, "ce": 0.007656258385395631, "kl": 0.16688690873229461, "context": 0.09109971160069108}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.3481321930885315, "loss_scale": 4.0, "retries": 0, "seconds": 215.7, "max_memory_gb": 17.17, "ce": 0.021813426807057112, "kl": 0.0161307665403001, "context": 0.02987881051376462}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 0.3957877457141876, "loss_scale": 4.0, "retries": 0, "seconds": 216.5, "max_memory_gb": 17.17, "ce": 0.0948153108911356, "kl": 0.030649738303509366, "context": 0.11014017730485648}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 1.461246371269226, "loss_scale": 4.0, "retries": 0, "seconds": 217.5, "max_memory_gb": 17.17, "ce": 0.06338811124442145, "kl": 0.0019864077635247668, "context": 0.06438131595496088}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 1.1472852230072021, "loss_scale": 4.0, "retries": 0, "seconds": 218.4, "max_memory_gb": 17.17, "ce": 0.028387966740410775, "kl": 0.4620836927642813, "context": 0.25942981615662575}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.3555854260921478, "loss_scale": 4.0, "retries": 0, "seconds": 219.4, "max_memory_gb": 17.17, "ce": 0.040423805854516104, "kl": 0.010381175037764478, "context": 0.04561439322424121}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 1.1600866317749023, "loss_scale": 4.0, "retries": 0, "seconds": 220.6, "max_memory_gb": 17.17, "ce": 0.0947896062862128, "kl": 0.06461056880652905, "context": 0.12709488708060235}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 1.2686296701431274, "loss_scale": 4.0, "retries": 0, "seconds": 221.7, "max_memory_gb": 17.17, "ce": 0.007160950393881649, "kl": 0.5414757546936926, "context": 0.27789882925571874}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 1.1716103553771973, "loss_scale": 4.0, "retries": 0, "seconds": 222.8, "max_memory_gb": 17.17, "ce": 0.044580146903172135, "kl": 0.07366458744081683, "context": 0.08141244068974629}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 1.31955087184906, "loss_scale": 4.0, "retries": 0, "seconds": 223.8, "max_memory_gb": 17.17, "ce": 0.04921663593268022, "kl": 0.05892759951893822, "context": 0.07868043475900777}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.6492602229118347, "loss_scale": 4.0, "retries": 0, "seconds": 224.8, "max_memory_gb": 17.17, "ce": 0.0394126575847622, "kl": 0.026891741199506214, "context": 0.05285852725501172}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.4815225601196289, "loss_scale": 4.0, "retries": 0, "seconds": 225.9, "max_memory_gb": 17.17, "ce": 0.013272042240714654, "kl": 0.11932689635432325, "context": 0.0729354887153022}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 1.8644969463348389, "loss_scale": 4.0, "retries": 0, "seconds": 226.9, "max_memory_gb": 17.17, "ce": 0.023141581536037847, "kl": 0.4284131914300815, "context": 0.23734817700460553}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 0.07004100829362869, "loss_scale": 4.0, "retries": 0, "seconds": 227.9, "max_memory_gb": 17.17, "ce": 0.004478089394979179, "kl": 0.0015421740172314458, "context": 0.005249176407232881}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.1517920047044754, "loss_scale": 4.0, "retries": 0, "seconds": 229.1, "max_memory_gb": 17.17, "ce": 0.012653469239012338, "kl": 0.013792046516755363, "context": 0.019549492018995807}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.8436248302459717, "loss_scale": 4.0, "retries": 0, "seconds": 230.1, "max_memory_gb": 17.17, "ce": 0.08776529479655437, "kl": 0.18632470583543181, "context": 0.18092764541506767}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.020579438656568527, "loss_scale": 4.0, "retries": 0, "seconds": 231.0, "max_memory_gb": 17.17, "ce": 0.0014024778065504506, "kl": 0.0007490161988243926, "context": 0.0017769858968677}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 1.4228875637054443, "loss_scale": 4.0, "retries": 0, "seconds": 232.1, "max_memory_gb": 17.17, "ce": 0.07030834461329505, "kl": 0.05765060651174281, "context": 0.0991336478618905}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 1.9078481197357178, "loss_scale": 4.0, "retries": 0, "seconds": 233.1, "max_memory_gb": 17.17, "ce": 0.12371573786367662, "kl": 0.2756402073428035, "context": 0.2615358391776681}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 1.7732691764831543, "loss_scale": 4.0, "retries": 0, "seconds": 234.2, "max_memory_gb": 17.17, "ce": 0.7831333820213331, "kl": 0.013134085478668567, "context": 0.7897004067344824}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.1081748902797699, "loss_scale": 4.0, "retries": 0, "seconds": 235.2, "max_memory_gb": 17.17, "ce": 0.010460046905791387, "kl": 0.005834441351908026, "context": 0.01337726782367099}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.8762138485908508, "loss_scale": 4.0, "retries": 0, "seconds": 236.2, "max_memory_gb": 17.17, "ce": 0.06056963940500282, "kl": 0.0013779909568256699, "context": 0.06125863455235958}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.27097761631011963, "loss_scale": 4.0, "retries": 0, "seconds": 237.3, "max_memory_gb": 17.17, "ce": 0.0031072615820448846, "kl": 0.05530617929798609, "context": 0.03076035127742216}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 1.0111048221588135, "loss_scale": 4.0, "retries": 0, "seconds": 238.3, "max_memory_gb": 17.17, "ce": 0.025211717787897214, "kl": 0.46816091103391955, "context": 0.25929217430530116}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 1.2925922870635986, "loss_scale": 4.0, "retries": 0, "seconds": 239.3, "max_memory_gb": 17.17, "ce": 0.10504580757697113, "kl": 0.11361531228430977, "context": 0.16185346184647642}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.4573967754840851, "loss_scale": 4.0, "retries": 0, "seconds": 240.4, "max_memory_gb": 17.17, "ce": 0.13830618464271538, "kl": 0.041678925868836814, "context": 0.1591456439346075}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.3272702693939209, "loss_scale": 4.0, "retries": 0, "seconds": 241.4, "max_memory_gb": 17.17, "ce": 0.05128268041880801, "kl": 0.012593634633958573, "context": 0.05757949926191941}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 1.3537335395812988, "loss_scale": 4.0, "retries": 0, "seconds": 242.3, "max_memory_gb": 17.17, "ce": 0.07252705027349293, "kl": 0.011872480812598951, "context": 0.07846328787854873}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 0.8781104683876038, "loss_scale": 4.0, "retries": 0, "seconds": 243.3, "max_memory_gb": 17.17, "ce": 0.09213062559138052, "kl": 0.08379925529243337, "context": 0.134030260698637}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.2518909275531769, "loss_scale": 4.0, "retries": 0, "seconds": 244.2, "max_memory_gb": 17.17, "ce": 0.03622202936094254, "kl": 0.004780973085871665, "context": 0.03861251490889117}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.994924783706665, "loss_scale": 4.0, "retries": 0, "seconds": 245.2, "max_memory_gb": 17.17, "ce": 0.19821723099448718, "kl": 0.011451328537077643, "context": 0.20394290203694254}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.9987691640853882, "loss_scale": 4.0, "retries": 0, "seconds": 246.0, "max_memory_gb": 17.17, "ce": 0.07354955363553017, "kl": 0.06911869297618978, "context": 0.10810890473658219}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.02981702797114849, "loss_scale": 4.0, "retries": 0, "seconds": 246.9, "max_memory_gb": 17.17, "ce": 0.002195052438764833, "kl": 0.0010794689876547636, "context": 0.002734786889050156}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.23014293611049652, "loss_scale": 4.0, "retries": 0, "seconds": 247.9, "max_memory_gb": 17.17, "ce": 0.022119493049103767, "kl": 0.01322549877295387, "context": 0.02873224241193384}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.9271966814994812, "loss_scale": 4.0, "retries": 0, "seconds": 248.8, "max_memory_gb": 17.17, "ce": 0.06451785651734099, "kl": 0.046902758826036006, "context": 0.08796923430054449}

runs/fixed_internvl/context/Full_s13/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Full_s13/adapter/adapter_config.json

@@ -0,0 +1,556 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.0.mlp.down_proj": 32,
+    "model.language_model.layers.0.mlp.gate_proj": 32,
+    "model.language_model.layers.0.mlp.up_proj": 32,
+    "model.language_model.layers.0.self_attn.k_proj": 32,
+    "model.language_model.layers.0.self_attn.o_proj": 32,
+    "model.language_model.layers.0.self_attn.q_proj": 32,
+    "model.language_model.layers.0.self_attn.v_proj": 32,
+    "model.language_model.layers.1.mlp.down_proj": 32,
+    "model.language_model.layers.1.mlp.gate_proj": 32,
+    "model.language_model.layers.1.mlp.up_proj": 32,
+    "model.language_model.layers.1.self_attn.k_proj": 32,
+    "model.language_model.layers.1.self_attn.o_proj": 32,
+    "model.language_model.layers.1.self_attn.q_proj": 32,
+    "model.language_model.layers.1.self_attn.v_proj": 32,
+    "model.language_model.layers.10.mlp.down_proj": 32,
+    "model.language_model.layers.10.mlp.gate_proj": 32,
+    "model.language_model.layers.10.mlp.up_proj": 32,
+    "model.language_model.layers.10.self_attn.k_proj": 32,
+    "model.language_model.layers.10.self_attn.o_proj": 32,
+    "model.language_model.layers.10.self_attn.q_proj": 32,
+    "model.language_model.layers.10.self_attn.v_proj": 32,
+    "model.language_model.layers.11.mlp.down_proj": 32,
+    "model.language_model.layers.11.mlp.gate_proj": 32,
+    "model.language_model.layers.11.mlp.up_proj": 32,
+    "model.language_model.layers.11.self_attn.k_proj": 32,
+    "model.language_model.layers.11.self_attn.o_proj": 32,
+    "model.language_model.layers.11.self_attn.q_proj": 32,
+    "model.language_model.layers.11.self_attn.v_proj": 32,
+    "model.language_model.layers.12.mlp.down_proj": 32,
+    "model.language_model.layers.12.mlp.gate_proj": 32,
+    "model.language_model.layers.12.mlp.up_proj": 32,
+    "model.language_model.layers.12.self_attn.k_proj": 32,
+    "model.language_model.layers.12.self_attn.o_proj": 32,
+    "model.language_model.layers.12.self_attn.q_proj": 32,
+    "model.language_model.layers.12.self_attn.v_proj": 32,
+    "model.language_model.layers.13.mlp.down_proj": 32,
+    "model.language_model.layers.13.mlp.gate_proj": 32,
+    "model.language_model.layers.13.mlp.up_proj": 32,
+    "model.language_model.layers.13.self_attn.k_proj": 32,
+    "model.language_model.layers.13.self_attn.o_proj": 32,
+    "model.language_model.layers.13.self_attn.q_proj": 32,
+    "model.language_model.layers.13.self_attn.v_proj": 32,
+    "model.language_model.layers.14.mlp.down_proj": 32,
+    "model.language_model.layers.14.mlp.gate_proj": 32,
+    "model.language_model.layers.14.mlp.up_proj": 32,
+    "model.language_model.layers.14.self_attn.k_proj": 32,
+    "model.language_model.layers.14.self_attn.o_proj": 32,
+    "model.language_model.layers.14.self_attn.q_proj": 32,
+    "model.language_model.layers.14.self_attn.v_proj": 32,
+    "model.language_model.layers.15.mlp.down_proj": 32,
+    "model.language_model.layers.15.mlp.gate_proj": 32,
+    "model.language_model.layers.15.mlp.up_proj": 32,
+    "model.language_model.layers.15.self_attn.k_proj": 32,
+    "model.language_model.layers.15.self_attn.o_proj": 32,
+    "model.language_model.layers.15.self_attn.q_proj": 32,
+    "model.language_model.layers.15.self_attn.v_proj": 32,
+    "model.language_model.layers.16.mlp.down_proj": 32,
+    "model.language_model.layers.16.mlp.gate_proj": 32,
+    "model.language_model.layers.16.mlp.up_proj": 32,
+    "model.language_model.layers.16.self_attn.k_proj": 32,
+    "model.language_model.layers.16.self_attn.o_proj": 32,
+    "model.language_model.layers.16.self_attn.q_proj": 32,
+    "model.language_model.layers.16.self_attn.v_proj": 32,
+    "model.language_model.layers.17.mlp.down_proj": 32,
+    "model.language_model.layers.17.mlp.gate_proj": 32,
+    "model.language_model.layers.17.mlp.up_proj": 32,
+    "model.language_model.layers.17.self_attn.k_proj": 32,
+    "model.language_model.layers.17.self_attn.o_proj": 32,
+    "model.language_model.layers.17.self_attn.q_proj": 32,
+    "model.language_model.layers.17.self_attn.v_proj": 32,
+    "model.language_model.layers.18.mlp.down_proj": 32,
+    "model.language_model.layers.18.mlp.gate_proj": 32,
+    "model.language_model.layers.18.mlp.up_proj": 32,
+    "model.language_model.layers.18.self_attn.k_proj": 32,
+    "model.language_model.layers.18.self_attn.o_proj": 32,
+    "model.language_model.layers.18.self_attn.q_proj": 32,
+    "model.language_model.layers.18.self_attn.v_proj": 32,
+    "model.language_model.layers.19.mlp.down_proj": 32,
+    "model.language_model.layers.19.mlp.gate_proj": 32,
+    "model.language_model.layers.19.mlp.up_proj": 32,
+    "model.language_model.layers.19.self_attn.k_proj": 32,
+    "model.language_model.layers.19.self_attn.o_proj": 32,
+    "model.language_model.layers.19.self_attn.q_proj": 32,
+    "model.language_model.layers.19.self_attn.v_proj": 32,
+    "model.language_model.layers.2.mlp.down_proj": 32,
+    "model.language_model.layers.2.mlp.gate_proj": 32,
+    "model.language_model.layers.2.mlp.up_proj": 32,
+    "model.language_model.layers.2.self_attn.k_proj": 32,
+    "model.language_model.layers.2.self_attn.o_proj": 32,
+    "model.language_model.layers.2.self_attn.q_proj": 32,
+    "model.language_model.layers.2.self_attn.v_proj": 32,
+    "model.language_model.layers.20.mlp.down_proj": 32,
+    "model.language_model.layers.20.mlp.gate_proj": 32,
+    "model.language_model.layers.20.mlp.up_proj": 32,
+    "model.language_model.layers.20.self_attn.k_proj": 32,
+    "model.language_model.layers.20.self_attn.o_proj": 32,
+    "model.language_model.layers.20.self_attn.q_proj": 32,
+    "model.language_model.layers.20.self_attn.v_proj": 32,
+    "model.language_model.layers.21.mlp.down_proj": 32,
+    "model.language_model.layers.21.mlp.gate_proj": 32,
+    "model.language_model.layers.21.mlp.up_proj": 32,
+    "model.language_model.layers.21.self_attn.k_proj": 32,
+    "model.language_model.layers.21.self_attn.o_proj": 32,
+    "model.language_model.layers.21.self_attn.q_proj": 32,
+    "model.language_model.layers.21.self_attn.v_proj": 32,
+    "model.language_model.layers.22.mlp.down_proj": 32,
+    "model.language_model.layers.22.mlp.gate_proj": 32,
+    "model.language_model.layers.22.mlp.up_proj": 32,
+    "model.language_model.layers.22.self_attn.k_proj": 32,
+    "model.language_model.layers.22.self_attn.o_proj": 32,
+    "model.language_model.layers.22.self_attn.q_proj": 32,
+    "model.language_model.layers.22.self_attn.v_proj": 32,
+    "model.language_model.layers.23.mlp.down_proj": 32,
+    "model.language_model.layers.23.mlp.gate_proj": 32,
+    "model.language_model.layers.23.mlp.up_proj": 32,
+    "model.language_model.layers.23.self_attn.k_proj": 32,
+    "model.language_model.layers.23.self_attn.o_proj": 32,
+    "model.language_model.layers.23.self_attn.q_proj": 32,
+    "model.language_model.layers.23.self_attn.v_proj": 32,
+    "model.language_model.layers.24.mlp.down_proj": 32,
+    "model.language_model.layers.24.mlp.gate_proj": 32,
+    "model.language_model.layers.24.mlp.up_proj": 32,
+    "model.language_model.layers.24.self_attn.k_proj": 32,
+    "model.language_model.layers.24.self_attn.o_proj": 32,
+    "model.language_model.layers.24.self_attn.q_proj": 32,
+    "model.language_model.layers.24.self_attn.v_proj": 32,
+    "model.language_model.layers.25.mlp.down_proj": 32,
+    "model.language_model.layers.25.mlp.gate_proj": 32,
+    "model.language_model.layers.25.mlp.up_proj": 32,
+    "model.language_model.layers.25.self_attn.k_proj": 32,
+    "model.language_model.layers.25.self_attn.o_proj": 32,
+    "model.language_model.layers.25.self_attn.q_proj": 32,
+    "model.language_model.layers.25.self_attn.v_proj": 32,
+    "model.language_model.layers.26.mlp.down_proj": 32,
+    "model.language_model.layers.26.mlp.gate_proj": 32,
+    "model.language_model.layers.26.mlp.up_proj": 32,
+    "model.language_model.layers.26.self_attn.k_proj": 32,
+    "model.language_model.layers.26.self_attn.o_proj": 32,
+    "model.language_model.layers.26.self_attn.q_proj": 32,
+    "model.language_model.layers.26.self_attn.v_proj": 32,
+    "model.language_model.layers.27.mlp.down_proj": 32,
+    "model.language_model.layers.27.mlp.gate_proj": 32,
+    "model.language_model.layers.27.mlp.up_proj": 32,
+    "model.language_model.layers.27.self_attn.k_proj": 32,
+    "model.language_model.layers.27.self_attn.o_proj": 32,
+    "model.language_model.layers.27.self_attn.q_proj": 32,
+    "model.language_model.layers.27.self_attn.v_proj": 32,
+    "model.language_model.layers.28.mlp.down_proj": 32,
+    "model.language_model.layers.28.mlp.gate_proj": 32,
+    "model.language_model.layers.28.mlp.up_proj": 32,
+    "model.language_model.layers.28.self_attn.k_proj": 32,
+    "model.language_model.layers.28.self_attn.o_proj": 32,
+    "model.language_model.layers.28.self_attn.q_proj": 32,
+    "model.language_model.layers.28.self_attn.v_proj": 32,
+    "model.language_model.layers.29.mlp.down_proj": 32,
+    "model.language_model.layers.29.mlp.gate_proj": 32,
+    "model.language_model.layers.29.mlp.up_proj": 32,
+    "model.language_model.layers.29.self_attn.k_proj": 32,
+    "model.language_model.layers.29.self_attn.o_proj": 32,
+    "model.language_model.layers.29.self_attn.q_proj": 32,
+    "model.language_model.layers.29.self_attn.v_proj": 32,
+    "model.language_model.layers.3.mlp.down_proj": 32,
+    "model.language_model.layers.3.mlp.gate_proj": 32,
+    "model.language_model.layers.3.mlp.up_proj": 32,
+    "model.language_model.layers.3.self_attn.k_proj": 32,
+    "model.language_model.layers.3.self_attn.o_proj": 32,
+    "model.language_model.layers.3.self_attn.q_proj": 32,
+    "model.language_model.layers.3.self_attn.v_proj": 32,
+    "model.language_model.layers.30.mlp.down_proj": 32,
+    "model.language_model.layers.30.mlp.gate_proj": 32,
+    "model.language_model.layers.30.mlp.up_proj": 32,
+    "model.language_model.layers.30.self_attn.k_proj": 32,
+    "model.language_model.layers.30.self_attn.o_proj": 32,
+    "model.language_model.layers.30.self_attn.q_proj": 32,
+    "model.language_model.layers.30.self_attn.v_proj": 32,
+    "model.language_model.layers.31.mlp.down_proj": 32,
+    "model.language_model.layers.31.mlp.gate_proj": 32,
+    "model.language_model.layers.31.mlp.up_proj": 32,
+    "model.language_model.layers.31.self_attn.k_proj": 32,
+    "model.language_model.layers.31.self_attn.o_proj": 32,
+    "model.language_model.layers.31.self_attn.q_proj": 32,
+    "model.language_model.layers.31.self_attn.v_proj": 32,
+    "model.language_model.layers.32.mlp.down_proj": 32,
+    "model.language_model.layers.32.mlp.gate_proj": 32,
+    "model.language_model.layers.32.mlp.up_proj": 32,
+    "model.language_model.layers.32.self_attn.k_proj": 32,
+    "model.language_model.layers.32.self_attn.o_proj": 32,
+    "model.language_model.layers.32.self_attn.q_proj": 32,
+    "model.language_model.layers.32.self_attn.v_proj": 32,
+    "model.language_model.layers.33.mlp.down_proj": 32,
+    "model.language_model.layers.33.mlp.gate_proj": 32,
+    "model.language_model.layers.33.mlp.up_proj": 32,
+    "model.language_model.layers.33.self_attn.k_proj": 32,
+    "model.language_model.layers.33.self_attn.o_proj": 32,
+    "model.language_model.layers.33.self_attn.q_proj": 32,
+    "model.language_model.layers.33.self_attn.v_proj": 32,
+    "model.language_model.layers.34.mlp.down_proj": 32,
+    "model.language_model.layers.34.mlp.gate_proj": 32,
+    "model.language_model.layers.34.mlp.up_proj": 32,
+    "model.language_model.layers.34.self_attn.k_proj": 32,
+    "model.language_model.layers.34.self_attn.o_proj": 32,
+    "model.language_model.layers.34.self_attn.q_proj": 32,
+    "model.language_model.layers.34.self_attn.v_proj": 32,
+    "model.language_model.layers.35.mlp.down_proj": 32,
+    "model.language_model.layers.35.mlp.gate_proj": 32,
+    "model.language_model.layers.35.mlp.up_proj": 32,
+    "model.language_model.layers.35.self_attn.k_proj": 32,
+    "model.language_model.layers.35.self_attn.o_proj": 32,
+    "model.language_model.layers.35.self_attn.q_proj": 32,
+    "model.language_model.layers.35.self_attn.v_proj": 32,
+    "model.language_model.layers.4.mlp.down_proj": 32,
+    "model.language_model.layers.4.mlp.gate_proj": 32,
+    "model.language_model.layers.4.mlp.up_proj": 32,
+    "model.language_model.layers.4.self_attn.k_proj": 32,
+    "model.language_model.layers.4.self_attn.o_proj": 32,
+    "model.language_model.layers.4.self_attn.q_proj": 32,
+    "model.language_model.layers.4.self_attn.v_proj": 32,
+    "model.language_model.layers.5.mlp.down_proj": 32,
+    "model.language_model.layers.5.mlp.gate_proj": 32,
+    "model.language_model.layers.5.mlp.up_proj": 32,
+    "model.language_model.layers.5.self_attn.k_proj": 32,
+    "model.language_model.layers.5.self_attn.o_proj": 32,
+    "model.language_model.layers.5.self_attn.q_proj": 32,
+    "model.language_model.layers.5.self_attn.v_proj": 32,
+    "model.language_model.layers.6.mlp.down_proj": 32,
+    "model.language_model.layers.6.mlp.gate_proj": 32,
+    "model.language_model.layers.6.mlp.up_proj": 32,
+    "model.language_model.layers.6.self_attn.k_proj": 32,
+    "model.language_model.layers.6.self_attn.o_proj": 32,
+    "model.language_model.layers.6.self_attn.q_proj": 32,
+    "model.language_model.layers.6.self_attn.v_proj": 32,
+    "model.language_model.layers.7.mlp.down_proj": 32,
+    "model.language_model.layers.7.mlp.gate_proj": 32,
+    "model.language_model.layers.7.mlp.up_proj": 32,
+    "model.language_model.layers.7.self_attn.k_proj": 32,
+    "model.language_model.layers.7.self_attn.o_proj": 32,
+    "model.language_model.layers.7.self_attn.q_proj": 32,
+    "model.language_model.layers.7.self_attn.v_proj": 32,
+    "model.language_model.layers.8.mlp.down_proj": 32,
+    "model.language_model.layers.8.mlp.gate_proj": 32,
+    "model.language_model.layers.8.mlp.up_proj": 32,
+    "model.language_model.layers.8.self_attn.k_proj": 32,
+    "model.language_model.layers.8.self_attn.o_proj": 32,
+    "model.language_model.layers.8.self_attn.q_proj": 32,
+    "model.language_model.layers.8.self_attn.v_proj": 32,
+    "model.language_model.layers.9.mlp.down_proj": 32,
+    "model.language_model.layers.9.mlp.gate_proj": 32,
+    "model.language_model.layers.9.mlp.up_proj": 32,
+    "model.language_model.layers.9.self_attn.k_proj": 32,
+    "model.language_model.layers.9.self_attn.o_proj": 32,
+    "model.language_model.layers.9.self_attn.q_proj": 32,
+    "model.language_model.layers.9.self_attn.v_proj": 32
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.0.mlp.down_proj": 16,
+    "model.language_model.layers.0.mlp.gate_proj": 16,
+    "model.language_model.layers.0.mlp.up_proj": 16,
+    "model.language_model.layers.0.self_attn.k_proj": 16,
+    "model.language_model.layers.0.self_attn.o_proj": 16,
+    "model.language_model.layers.0.self_attn.q_proj": 16,
+    "model.language_model.layers.0.self_attn.v_proj": 16,
+    "model.language_model.layers.1.mlp.down_proj": 16,
+    "model.language_model.layers.1.mlp.gate_proj": 16,
+    "model.language_model.layers.1.mlp.up_proj": 16,
+    "model.language_model.layers.1.self_attn.k_proj": 16,
+    "model.language_model.layers.1.self_attn.o_proj": 16,
+    "model.language_model.layers.1.self_attn.q_proj": 16,
+    "model.language_model.layers.1.self_attn.v_proj": 16,
+    "model.language_model.layers.10.mlp.down_proj": 16,
+    "model.language_model.layers.10.mlp.gate_proj": 16,
+    "model.language_model.layers.10.mlp.up_proj": 16,
+    "model.language_model.layers.10.self_attn.k_proj": 16,
+    "model.language_model.layers.10.self_attn.o_proj": 16,
+    "model.language_model.layers.10.self_attn.q_proj": 16,
+    "model.language_model.layers.10.self_attn.v_proj": 16,
+    "model.language_model.layers.11.mlp.down_proj": 16,
+    "model.language_model.layers.11.mlp.gate_proj": 16,
+    "model.language_model.layers.11.mlp.up_proj": 16,
+    "model.language_model.layers.11.self_attn.k_proj": 16,
+    "model.language_model.layers.11.self_attn.o_proj": 16,
+    "model.language_model.layers.11.self_attn.q_proj": 16,
+    "model.language_model.layers.11.self_attn.v_proj": 16,
+    "model.language_model.layers.12.mlp.down_proj": 16,
+    "model.language_model.layers.12.mlp.gate_proj": 16,
+    "model.language_model.layers.12.mlp.up_proj": 16,
+    "model.language_model.layers.12.self_attn.k_proj": 16,
+    "model.language_model.layers.12.self_attn.o_proj": 16,
+    "model.language_model.layers.12.self_attn.q_proj": 16,
+    "model.language_model.layers.12.self_attn.v_proj": 16,
+    "model.language_model.layers.13.mlp.down_proj": 16,
+    "model.language_model.layers.13.mlp.gate_proj": 16,
+    "model.language_model.layers.13.mlp.up_proj": 16,
+    "model.language_model.layers.13.self_attn.k_proj": 16,
+    "model.language_model.layers.13.self_attn.o_proj": 16,
+    "model.language_model.layers.13.self_attn.q_proj": 16,
+    "model.language_model.layers.13.self_attn.v_proj": 16,
+    "model.language_model.layers.14.mlp.down_proj": 16,
+    "model.language_model.layers.14.mlp.gate_proj": 16,
+    "model.language_model.layers.14.mlp.up_proj": 16,
+    "model.language_model.layers.14.self_attn.k_proj": 16,
+    "model.language_model.layers.14.self_attn.o_proj": 16,
+    "model.language_model.layers.14.self_attn.q_proj": 16,
+    "model.language_model.layers.14.self_attn.v_proj": 16,
+    "model.language_model.layers.15.mlp.down_proj": 16,
+    "model.language_model.layers.15.mlp.gate_proj": 16,
+    "model.language_model.layers.15.mlp.up_proj": 16,
+    "model.language_model.layers.15.self_attn.k_proj": 16,
+    "model.language_model.layers.15.self_attn.o_proj": 16,
+    "model.language_model.layers.15.self_attn.q_proj": 16,
+    "model.language_model.layers.15.self_attn.v_proj": 16,
+    "model.language_model.layers.16.mlp.down_proj": 16,
+    "model.language_model.layers.16.mlp.gate_proj": 16,
+    "model.language_model.layers.16.mlp.up_proj": 16,
+    "model.language_model.layers.16.self_attn.k_proj": 16,
+    "model.language_model.layers.16.self_attn.o_proj": 16,
+    "model.language_model.layers.16.self_attn.q_proj": 16,
+    "model.language_model.layers.16.self_attn.v_proj": 16,
+    "model.language_model.layers.17.mlp.down_proj": 16,
+    "model.language_model.layers.17.mlp.gate_proj": 16,
+    "model.language_model.layers.17.mlp.up_proj": 16,
+    "model.language_model.layers.17.self_attn.k_proj": 16,
+    "model.language_model.layers.17.self_attn.o_proj": 16,
+    "model.language_model.layers.17.self_attn.q_proj": 16,
+    "model.language_model.layers.17.self_attn.v_proj": 16,
+    "model.language_model.layers.18.mlp.down_proj": 16,
+    "model.language_model.layers.18.mlp.gate_proj": 16,
+    "model.language_model.layers.18.mlp.up_proj": 16,
+    "model.language_model.layers.18.self_attn.k_proj": 16,
+    "model.language_model.layers.18.self_attn.o_proj": 16,
+    "model.language_model.layers.18.self_attn.q_proj": 16,
+    "model.language_model.layers.18.self_attn.v_proj": 16,
+    "model.language_model.layers.19.mlp.down_proj": 16,
+    "model.language_model.layers.19.mlp.gate_proj": 16,
+    "model.language_model.layers.19.mlp.up_proj": 16,
+    "model.language_model.layers.19.self_attn.k_proj": 16,
+    "model.language_model.layers.19.self_attn.o_proj": 16,
+    "model.language_model.layers.19.self_attn.q_proj": 16,
+    "model.language_model.layers.19.self_attn.v_proj": 16,
+    "model.language_model.layers.2.mlp.down_proj": 16,
+    "model.language_model.layers.2.mlp.gate_proj": 16,
+    "model.language_model.layers.2.mlp.up_proj": 16,
+    "model.language_model.layers.2.self_attn.k_proj": 16,
+    "model.language_model.layers.2.self_attn.o_proj": 16,
+    "model.language_model.layers.2.self_attn.q_proj": 16,
+    "model.language_model.layers.2.self_attn.v_proj": 16,
+    "model.language_model.layers.20.mlp.down_proj": 16,
+    "model.language_model.layers.20.mlp.gate_proj": 16,
+    "model.language_model.layers.20.mlp.up_proj": 16,
+    "model.language_model.layers.20.self_attn.k_proj": 16,
+    "model.language_model.layers.20.self_attn.o_proj": 16,
+    "model.language_model.layers.20.self_attn.q_proj": 16,
+    "model.language_model.layers.20.self_attn.v_proj": 16,
+    "model.language_model.layers.21.mlp.down_proj": 16,
+    "model.language_model.layers.21.mlp.gate_proj": 16,
+    "model.language_model.layers.21.mlp.up_proj": 16,
+    "model.language_model.layers.21.self_attn.k_proj": 16,
+    "model.language_model.layers.21.self_attn.o_proj": 16,
+    "model.language_model.layers.21.self_attn.q_proj": 16,
+    "model.language_model.layers.21.self_attn.v_proj": 16,
+    "model.language_model.layers.22.mlp.down_proj": 16,
+    "model.language_model.layers.22.mlp.gate_proj": 16,
+    "model.language_model.layers.22.mlp.up_proj": 16,
+    "model.language_model.layers.22.self_attn.k_proj": 16,
+    "model.language_model.layers.22.self_attn.o_proj": 16,
+    "model.language_model.layers.22.self_attn.q_proj": 16,
+    "model.language_model.layers.22.self_attn.v_proj": 16,
+    "model.language_model.layers.23.mlp.down_proj": 16,
+    "model.language_model.layers.23.mlp.gate_proj": 16,
+    "model.language_model.layers.23.mlp.up_proj": 16,
+    "model.language_model.layers.23.self_attn.k_proj": 16,
+    "model.language_model.layers.23.self_attn.o_proj": 16,
+    "model.language_model.layers.23.self_attn.q_proj": 16,
+    "model.language_model.layers.23.self_attn.v_proj": 16,
+    "model.language_model.layers.24.mlp.down_proj": 16,
+    "model.language_model.layers.24.mlp.gate_proj": 16,
+    "model.language_model.layers.24.mlp.up_proj": 16,
+    "model.language_model.layers.24.self_attn.k_proj": 16,
+    "model.language_model.layers.24.self_attn.o_proj": 16,
+    "model.language_model.layers.24.self_attn.q_proj": 16,
+    "model.language_model.layers.24.self_attn.v_proj": 16,
+    "model.language_model.layers.25.mlp.down_proj": 16,
+    "model.language_model.layers.25.mlp.gate_proj": 16,
+    "model.language_model.layers.25.mlp.up_proj": 16,
+    "model.language_model.layers.25.self_attn.k_proj": 16,
+    "model.language_model.layers.25.self_attn.o_proj": 16,
+    "model.language_model.layers.25.self_attn.q_proj": 16,
+    "model.language_model.layers.25.self_attn.v_proj": 16,
+    "model.language_model.layers.26.mlp.down_proj": 16,
+    "model.language_model.layers.26.mlp.gate_proj": 16,
+    "model.language_model.layers.26.mlp.up_proj": 16,
+    "model.language_model.layers.26.self_attn.k_proj": 16,
+    "model.language_model.layers.26.self_attn.o_proj": 16,
+    "model.language_model.layers.26.self_attn.q_proj": 16,
+    "model.language_model.layers.26.self_attn.v_proj": 16,
+    "model.language_model.layers.27.mlp.down_proj": 16,
+    "model.language_model.layers.27.mlp.gate_proj": 16,
+    "model.language_model.layers.27.mlp.up_proj": 16,
+    "model.language_model.layers.27.self_attn.k_proj": 16,
+    "model.language_model.layers.27.self_attn.o_proj": 16,
+    "model.language_model.layers.27.self_attn.q_proj": 16,
+    "model.language_model.layers.27.self_attn.v_proj": 16,
+    "model.language_model.layers.28.mlp.down_proj": 16,
+    "model.language_model.layers.28.mlp.gate_proj": 16,
+    "model.language_model.layers.28.mlp.up_proj": 16,
+    "model.language_model.layers.28.self_attn.k_proj": 16,
+    "model.language_model.layers.28.self_attn.o_proj": 16,
+    "model.language_model.layers.28.self_attn.q_proj": 16,
+    "model.language_model.layers.28.self_attn.v_proj": 16,
+    "model.language_model.layers.29.mlp.down_proj": 16,
+    "model.language_model.layers.29.mlp.gate_proj": 16,
+    "model.language_model.layers.29.mlp.up_proj": 16,
+    "model.language_model.layers.29.self_attn.k_proj": 16,
+    "model.language_model.layers.29.self_attn.o_proj": 16,
+    "model.language_model.layers.29.self_attn.q_proj": 16,
+    "model.language_model.layers.29.self_attn.v_proj": 16,
+    "model.language_model.layers.3.mlp.down_proj": 16,
+    "model.language_model.layers.3.mlp.gate_proj": 16,
+    "model.language_model.layers.3.mlp.up_proj": 16,
+    "model.language_model.layers.3.self_attn.k_proj": 16,
+    "model.language_model.layers.3.self_attn.o_proj": 16,
+    "model.language_model.layers.3.self_attn.q_proj": 16,
+    "model.language_model.layers.3.self_attn.v_proj": 16,
+    "model.language_model.layers.30.mlp.down_proj": 16,
+    "model.language_model.layers.30.mlp.gate_proj": 16,
+    "model.language_model.layers.30.mlp.up_proj": 16,
+    "model.language_model.layers.30.self_attn.k_proj": 16,
+    "model.language_model.layers.30.self_attn.o_proj": 16,
+    "model.language_model.layers.30.self_attn.q_proj": 16,
+    "model.language_model.layers.30.self_attn.v_proj": 16,
+    "model.language_model.layers.31.mlp.down_proj": 16,
+    "model.language_model.layers.31.mlp.gate_proj": 16,
+    "model.language_model.layers.31.mlp.up_proj": 16,
+    "model.language_model.layers.31.self_attn.k_proj": 16,
+    "model.language_model.layers.31.self_attn.o_proj": 16,
+    "model.language_model.layers.31.self_attn.q_proj": 16,
+    "model.language_model.layers.31.self_attn.v_proj": 16,
+    "model.language_model.layers.32.mlp.down_proj": 16,
+    "model.language_model.layers.32.mlp.gate_proj": 16,
+    "model.language_model.layers.32.mlp.up_proj": 16,
+    "model.language_model.layers.32.self_attn.k_proj": 16,
+    "model.language_model.layers.32.self_attn.o_proj": 16,
+    "model.language_model.layers.32.self_attn.q_proj": 16,
+    "model.language_model.layers.32.self_attn.v_proj": 16,
+    "model.language_model.layers.33.mlp.down_proj": 16,
+    "model.language_model.layers.33.mlp.gate_proj": 16,
+    "model.language_model.layers.33.mlp.up_proj": 16,
+    "model.language_model.layers.33.self_attn.k_proj": 16,
+    "model.language_model.layers.33.self_attn.o_proj": 16,
+    "model.language_model.layers.33.self_attn.q_proj": 16,
+    "model.language_model.layers.33.self_attn.v_proj": 16,
+    "model.language_model.layers.34.mlp.down_proj": 16,
+    "model.language_model.layers.34.mlp.gate_proj": 16,
+    "model.language_model.layers.34.mlp.up_proj": 16,
+    "model.language_model.layers.34.self_attn.k_proj": 16,
+    "model.language_model.layers.34.self_attn.o_proj": 16,
+    "model.language_model.layers.34.self_attn.q_proj": 16,
+    "model.language_model.layers.34.self_attn.v_proj": 16,
+    "model.language_model.layers.35.mlp.down_proj": 16,
+    "model.language_model.layers.35.mlp.gate_proj": 16,
+    "model.language_model.layers.35.mlp.up_proj": 16,
+    "model.language_model.layers.35.self_attn.k_proj": 16,
+    "model.language_model.layers.35.self_attn.o_proj": 16,
+    "model.language_model.layers.35.self_attn.q_proj": 16,
+    "model.language_model.layers.35.self_attn.v_proj": 16,
+    "model.language_model.layers.4.mlp.down_proj": 16,
+    "model.language_model.layers.4.mlp.gate_proj": 16,
+    "model.language_model.layers.4.mlp.up_proj": 16,
+    "model.language_model.layers.4.self_attn.k_proj": 16,
+    "model.language_model.layers.4.self_attn.o_proj": 16,
+    "model.language_model.layers.4.self_attn.q_proj": 16,
+    "model.language_model.layers.4.self_attn.v_proj": 16,
+    "model.language_model.layers.5.mlp.down_proj": 16,
+    "model.language_model.layers.5.mlp.gate_proj": 16,
+    "model.language_model.layers.5.mlp.up_proj": 16,
+    "model.language_model.layers.5.self_attn.k_proj": 16,
+    "model.language_model.layers.5.self_attn.o_proj": 16,
+    "model.language_model.layers.5.self_attn.q_proj": 16,
+    "model.language_model.layers.5.self_attn.v_proj": 16,
+    "model.language_model.layers.6.mlp.down_proj": 16,
+    "model.language_model.layers.6.mlp.gate_proj": 16,
+    "model.language_model.layers.6.mlp.up_proj": 16,
+    "model.language_model.layers.6.self_attn.k_proj": 16,
+    "model.language_model.layers.6.self_attn.o_proj": 16,
+    "model.language_model.layers.6.self_attn.q_proj": 16,
+    "model.language_model.layers.6.self_attn.v_proj": 16,
+    "model.language_model.layers.7.mlp.down_proj": 16,
+    "model.language_model.layers.7.mlp.gate_proj": 16,
+    "model.language_model.layers.7.mlp.up_proj": 16,
+    "model.language_model.layers.7.self_attn.k_proj": 16,
+    "model.language_model.layers.7.self_attn.o_proj": 16,
+    "model.language_model.layers.7.self_attn.q_proj": 16,
+    "model.language_model.layers.7.self_attn.v_proj": 16,
+    "model.language_model.layers.8.mlp.down_proj": 16,
+    "model.language_model.layers.8.mlp.gate_proj": 16,
+    "model.language_model.layers.8.mlp.up_proj": 16,
+    "model.language_model.layers.8.self_attn.k_proj": 16,
+    "model.language_model.layers.8.self_attn.o_proj": 16,
+    "model.language_model.layers.8.self_attn.q_proj": 16,
+    "model.language_model.layers.8.self_attn.v_proj": 16,
+    "model.language_model.layers.9.mlp.down_proj": 16,
+    "model.language_model.layers.9.mlp.gate_proj": 16,
+    "model.language_model.layers.9.mlp.up_proj": 16,
+    "model.language_model.layers.9.self_attn.k_proj": 16,
+    "model.language_model.layers.9.self_attn.o_proj": 16,
+    "model.language_model.layers.9.self_attn.q_proj": 16,
+    "model.language_model.layers.9.self_attn.v_proj": 16
+  },
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "up_proj",
+    "self_attn.v_proj",
+    "o_proj",
+    "self_attn.q_proj",
+    "self_attn.k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Full_s13/config.json

@@ -0,0 +1,99 @@
+{
+  "behavior": "context",
+  "seed": 13,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Full",
+    "layers": [
+      0,
+      1,
+      2,
+      3,
+      4,
+      5,
+      6,
+      7,
+      8,
+      9,
+      10,
+      11,
+      12,
+      13,
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21,
+      22,
+      23,
+      24,
+      25,
+      26,
+      27,
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35
+    ],
+    "ranks": {
+      "0": 16,
+      "1": 16,
+      "2": 16,
+      "3": 16,
+      "4": 16,
+      "5": 16,
+      "6": 16,
+      "7": 16,
+      "8": 16,
+      "9": 16,
+      "10": 16,
+      "11": 16,
+      "12": 16,
+      "13": 16,
+      "14": 16,
+      "15": 16,
+      "16": 16,
+      "17": 16,
+      "18": 16,
+      "19": 16,
+      "20": 16,
+      "21": 16,
+      "22": 16,
+      "23": 16,
+      "24": 16,
+      "25": 16,
+      "26": 16,
+      "27": 16,
+      "28": 16,
+      "29": 16,
+      "30": 16,
+      "31": 16,
+      "32": 16,
+      "33": 16,
+      "34": 16,
+      "35": 16
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Full_s13/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 5.424587249755859, "loss_scale": 4.0, "retries": 0, "seconds": 3.0, "max_memory_gb": 16.79, "ce": 0.5227766726748087, "kl": 0.08654405153356493, "context": 0.5660487064160407}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 7.009312152862549, "loss_scale": 4.0, "retries": 0, "seconds": 4.7, "max_memory_gb": 17.12, "ce": 0.6356047712251893, "kl": 0.46623069849079, "context": 0.8687201264692703}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 20.191789627075195, "loss_scale": 4.0, "retries": 0, "seconds": 6.5, "max_memory_gb": 17.12, "ce": 1.0391803830862045, "kl": 0.06928583979606628, "context": 1.0738233104348183}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 9.162154197692871, "loss_scale": 4.0, "retries": 0, "seconds": 8.3, "max_memory_gb": 17.12, "ce": 0.3558672432336607, "kl": 0.4414534467678095, "context": 0.5765939876582706}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 0.19955886900424957, "loss_scale": 4.0, "retries": 0, "seconds": 10.2, "max_memory_gb": 17.12, "ce": 0.009434058216356789, "kl": 0.00846104388256208, "context": 0.013664579924807185}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 3.5183911323547363, "loss_scale": 4.0, "retries": 0, "seconds": 11.9, "max_memory_gb": 17.12, "ce": 0.18055486791126896, "kl": 0.07903478993466706, "context": 0.22007226136338431}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 5.285635948181152, "loss_scale": 4.0, "retries": 0, "seconds": 13.8, "max_memory_gb": 17.12, "ce": 0.1696822503599833, "kl": 0.07611831858662299, "context": 0.20774140918729245}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 2.5026986598968506, "loss_scale": 4.0, "retries": 0, "seconds": 15.6, "max_memory_gb": 17.12, "ce": 0.06257978267967701, "kl": 0.2126260765362531, "context": 0.16889282502233982}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 7.485344886779785, "loss_scale": 4.0, "retries": 0, "seconds": 17.6, "max_memory_gb": 17.12, "ce": 0.09227220248430967, "kl": 0.16222309359000064, "context": 0.17338375188410282}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 0.6577385663986206, "loss_scale": 4.0, "retries": 0, "seconds": 19.4, "max_memory_gb": 17.12, "ce": 0.005340514915587846, "kl": 0.02264450816437602, "context": 0.016662769427057356}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.16316789388656616, "loss_scale": 4.0, "retries": 0, "seconds": 21.3, "max_memory_gb": 17.12, "ce": 0.00573814404924633, "kl": 0.00415438136587909, "context": 0.007815334844053723}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 0.2353709489107132, "loss_scale": 4.0, "retries": 0, "seconds": 23.0, "max_memory_gb": 17.12, "ce": 0.009820478048368386, "kl": 0.002225418287252978, "context": 0.01093318713355984}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.06892294436693192, "loss_scale": 4.0, "retries": 0, "seconds": 24.9, "max_memory_gb": 17.12, "ce": 0.0018064889936795225, "kl": 0.0004873835641774349, "context": 0.0020501807921391446}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 2.63189435005188, "loss_scale": 4.0, "retries": 0, "seconds": 26.6, "max_memory_gb": 17.12, "ce": 0.053025833156425506, "kl": 0.7490342447999865, "context": 0.4275429677218199}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 1.0722708702087402, "loss_scale": 4.0, "retries": 0, "seconds": 28.4, "max_memory_gb": 17.12, "ce": 0.008515537188259259, "kl": 0.3451146876882092, "context": 0.18107287804195948}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 0.03979145362973213, "loss_scale": 4.0, "retries": 0, "seconds": 30.2, "max_memory_gb": 17.12, "ce": 0.002021537638938753, "kl": 0.0007540913150023698, "context": 0.0023985832376638427}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 2.1016933917999268, "loss_scale": 4.0, "retries": 0, "seconds": 32.0, "max_memory_gb": 17.12, "ce": 0.024064863901003264, "kl": 0.7867922405930585, "context": 0.41746099188458174}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.8810983300209045, "loss_scale": 4.0, "retries": 0, "seconds": 33.7, "max_memory_gb": 17.13, "ce": 0.006730248234816827, "kl": 0.25938572040831787, "context": 0.13642310675641056}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.4912026524543762, "loss_scale": 4.0, "retries": 0, "seconds": 35.6, "max_memory_gb": 17.13, "ce": 0.004577323589728621, "kl": 0.08514375708000443, "context": 0.047149202508080634}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.1083262488245964, "loss_scale": 4.0, "retries": 0, "seconds": 37.3, "max_memory_gb": 17.13, "ce": 0.0008890613153198501, "kl": 0.007304282336917822, "context": 0.004541202579275705}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.04138277471065521, "loss_scale": 4.0, "retries": 0, "seconds": 39.2, "max_memory_gb": 17.13, "ce": 0.0005660811939378618, "kl": 0.0028678575881713186, "context": 0.0020000099739263533}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 1.7083793878555298, "loss_scale": 4.0, "retries": 0, "seconds": 41.0, "max_memory_gb": 17.13, "ce": 0.05979195245890878, "kl": 0.06165873514873965, "context": 0.0906213195703458}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 3.0094966888427734, "loss_scale": 4.0, "retries": 0, "seconds": 42.9, "max_memory_gb": 17.13, "ce": 0.0003123780006717425, "kl": 0.3523157566429518, "context": 0.1764702573636896}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.5332714319229126, "loss_scale": 4.0, "retries": 0, "seconds": 44.6, "max_memory_gb": 17.13, "ce": 0.007087269314069999, "kl": 0.03827214445118443, "context": 0.02622334179613972}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 2.8886046409606934, "loss_scale": 4.0, "retries": 0, "seconds": 46.3, "max_memory_gb": 17.13, "ce": 0.1823803555671475, "kl": 0.03616596118081361, "context": 0.20046333613572642}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 1.2145136594772339, "loss_scale": 4.0, "retries": 0, "seconds": 48.3, "max_memory_gb": 17.13, "ce": 0.04416358650269103, "kl": 0.5360694387636613, "context": 0.31219832081114873}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 1.5157434940338135, "loss_scale": 4.0, "retries": 0, "seconds": 50.4, "max_memory_gb": 17.13, "ce": 0.07802625276235631, "kl": 0.2840794892581471, "context": 0.2200659903464839}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 2.1026597023010254, "loss_scale": 4.0, "retries": 0, "seconds": 52.2, "max_memory_gb": 17.13, "ce": 0.11621186958836915, "kl": 0.23149095964618027, "context": 0.23195734806358814}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 3.900766372680664, "loss_scale": 4.0, "retries": 0, "seconds": 53.9, "max_memory_gb": 17.13, "ce": 0.5038846182869747, "kl": 0.1386234980309382, "context": 0.5731963673606515}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.22697365283966064, "loss_scale": 4.0, "retries": 0, "seconds": 55.7, "max_memory_gb": 17.13, "ce": 0.003577716608560877, "kl": 0.009715399360857191, "context": 0.008435416111751692}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 6.864530086517334, "loss_scale": 4.0, "retries": 0, "seconds": 57.4, "max_memory_gb": 17.13, "ce": 0.6713635913620237, "kl": 0.0925882534502307, "context": 0.7176577192731202}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 1.0501266717910767, "loss_scale": 4.0, "retries": 0, "seconds": 59.2, "max_memory_gb": 17.13, "ce": 0.03339595231227577, "kl": 0.033371706726029515, "context": 0.05008180672302842}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 3.20111083984375, "loss_scale": 4.0, "retries": 0, "seconds": 60.9, "max_memory_gb": 17.13, "ce": 0.32481611624825746, "kl": 0.07365645002573729, "context": 0.36164433788508177}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.180325984954834, "loss_scale": 4.0, "retries": 0, "seconds": 62.7, "max_memory_gb": 17.13, "ce": 0.00784374144859612, "kl": 0.7232472255127504, "context": 0.3694673535064794}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.09901473671197891, "loss_scale": 4.0, "retries": 0, "seconds": 64.2, "max_memory_gb": 17.13, "ce": 0.003507658497255761, "kl": 0.0006205957301972376, "context": 0.0038179563598532695}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 2.253882646560669, "loss_scale": 4.0, "retries": 0, "seconds": 65.9, "max_memory_gb": 17.13, "ce": 0.21501696412451565, "kl": 0.036797451495658606, "context": 0.23341569618787616}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 1.9726786613464355, "loss_scale": 4.0, "retries": 0, "seconds": 67.6, "max_memory_gb": 17.13, "ce": 0.104461716269725, "kl": 0.3858063513180241, "context": 0.2973648870829493}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 1.4069726467132568, "loss_scale": 4.0, "retries": 0, "seconds": 69.5, "max_memory_gb": 17.13, "ce": 0.08816959260730073, "kl": 0.376405202478054, "context": 0.27637219708412886}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.139986276626587, "loss_scale": 4.0, "retries": 0, "seconds": 71.2, "max_memory_gb": 17.13, "ce": 0.06612914334982634, "kl": 0.00844337634407566, "context": 0.0703508352744393}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.38438117504119873, "loss_scale": 4.0, "retries": 0, "seconds": 73.1, "max_memory_gb": 17.13, "ce": 0.024569131899625063, "kl": 0.01685650256695226, "context": 0.03299738303758204}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 3.162874221801758, "loss_scale": 4.0, "retries": 0, "seconds": 74.8, "max_memory_gb": 17.13, "ce": 0.48303021397441626, "kl": 0.08012373885139823, "context": 0.5230920724570751}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 0.13186155259609222, "loss_scale": 4.0, "retries": 0, "seconds": 76.6, "max_memory_gb": 17.13, "ce": 0.00958656711736694, "kl": 0.000858962588608847, "context": 0.010016048297984526}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 0.10911085456609726, "loss_scale": 4.0, "retries": 0, "seconds": 78.4, "max_memory_gb": 17.13, "ce": 0.006577975582331419, "kl": 0.0009521301108179614, "context": 0.007054040732327849}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 11.829795837402344, "loss_scale": 4.0, "retries": 0, "seconds": 80.3, "max_memory_gb": 17.13, "ce": 0.15441789047326893, "kl": 0.7597605735936668, "context": 0.5342981905560009}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 4.011559009552002, "loss_scale": 4.0, "retries": 0, "seconds": 81.9, "max_memory_gb": 17.13, "ce": 0.46531292039435357, "kl": 0.1783636300242506, "context": 0.5544947377638891}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 1.1134005784988403, "loss_scale": 4.0, "retries": 0, "seconds": 83.6, "max_memory_gb": 17.13, "ce": 0.07387978356564417, "kl": 0.1288526901626028, "context": 0.13830612448509783}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.9363508820533752, "loss_scale": 4.0, "retries": 0, "seconds": 85.1, "max_memory_gb": 17.13, "ce": 0.03138572006719187, "kl": 0.13367232965538278, "context": 0.09822188399266452}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.09596098214387894, "loss_scale": 4.0, "retries": 0, "seconds": 86.8, "max_memory_gb": 17.13, "ce": 0.004866934265010059, "kl": 0.0030889849658706225, "context": 0.006411426700651646}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.27122762799263, "loss_scale": 4.0, "retries": 0, "seconds": 88.5, "max_memory_gb": 17.13, "ce": 0.012206344894366339, "kl": 0.027677061611029785, "context": 0.0260448751214426}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.8441938161849976, "loss_scale": 4.0, "retries": 0, "seconds": 90.3, "max_memory_gb": 17.13, "ce": 0.06441719745635055, "kl": 0.008978238598501775, "context": 0.06890631726128049}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 63.84067916870117, "loss_scale": 4.0, "retries": 0, "seconds": 92.0, "max_memory_gb": 17.13, "ce": 0.09910061900154687, "kl": 0.016788694620117894, "context": 0.107494970143307}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.4832775592803955, "loss_scale": 4.0, "retries": 0, "seconds": 93.8, "max_memory_gb": 17.13, "ce": 0.020171633223071694, "kl": 0.10424128902377561, "context": 0.0722922794520855}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 1.3514559268951416, "loss_scale": 4.0, "retries": 0, "seconds": 95.5, "max_memory_gb": 17.13, "ce": 0.03980078001040965, "kl": 0.24658849509432912, "context": 0.16309502837248147}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 2.1659321784973145, "loss_scale": 4.0, "retries": 0, "seconds": 97.3, "max_memory_gb": 17.13, "ce": 0.13500449451385066, "kl": 0.07698360919312108, "context": 0.17349629889940843}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 0.3358761966228485, "loss_scale": 4.0, "retries": 0, "seconds": 99.1, "max_memory_gb": 17.13, "ce": 0.022897057817317545, "kl": 0.011603854196437169, "context": 0.02869898424251005}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.5852829813957214, "loss_scale": 4.0, "retries": 0, "seconds": 100.8, "max_memory_gb": 17.13, "ce": 0.028738461842294782, "kl": 0.0775656831683591, "context": 0.06752130528911948}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.2368646115064621, "loss_scale": 4.0, "retries": 0, "seconds": 102.7, "max_memory_gb": 17.13, "ce": 0.009878769516944885, "kl": 0.01165201778349001, "context": 0.015704778430517763}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.03005366027355194, "loss_scale": 4.0, "retries": 0, "seconds": 104.4, "max_memory_gb": 17.13, "ce": 0.001630116908927448, "kl": 0.00023249829246196896, "context": 0.0017463660333305597}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 1.5371953248977661, "loss_scale": 4.0, "retries": 0, "seconds": 106.2, "max_memory_gb": 17.13, "ce": 0.138282203261042, "kl": 0.07240554421150591, "context": 0.17448496929137036}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 1.578467607498169, "loss_scale": 4.0, "retries": 0, "seconds": 107.9, "max_memory_gb": 17.13, "ce": 0.14672977337613702, "kl": 0.10318157431902364, "context": 0.198320560506545}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 1.1834657192230225, "loss_scale": 4.0, "retries": 0, "seconds": 109.8, "max_memory_gb": 17.13, "ce": 0.08707829762715846, "kl": 0.46738688010373153, "context": 0.32077173970174044}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.5337945222854614, "loss_scale": 4.0, "retries": 0, "seconds": 111.4, "max_memory_gb": 17.13, "ce": 0.03262635215651244, "kl": 0.010180906807363499, "context": 0.037716804654337466}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 1.366684913635254, "loss_scale": 4.0, "retries": 0, "seconds": 113.3, "max_memory_gb": 17.13, "ce": 0.09986423118971288, "kl": 0.10825092074810527, "context": 0.15398969082161784}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.28762853145599365, "loss_scale": 4.0, "retries": 0, "seconds": 115.0, "max_memory_gb": 17.13, "ce": 0.01750482799252495, "kl": 0.00788046681554988, "context": 0.02144506137119606}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 0.8278865218162537, "loss_scale": 4.0, "retries": 0, "seconds": 117.0, "max_memory_gb": 17.13, "ce": 0.13059037551283836, "kl": 0.10455948067829013, "context": 0.1828701151534915}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 1.8724730014801025, "loss_scale": 4.0, "retries": 0, "seconds": 118.6, "max_memory_gb": 17.13, "ce": 0.17643119977947208, "kl": 0.08137530505518953, "context": 0.21711885238619288}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.08641718327999115, "loss_scale": 4.0, "retries": 0, "seconds": 120.5, "max_memory_gb": 17.13, "ce": 0.005897684313822538, "kl": 0.005655840771851217, "context": 0.00872560495918151}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.3046899139881134, "loss_scale": 4.0, "retries": 0, "seconds": 122.3, "max_memory_gb": 17.13, "ce": 0.010717209297581576, "kl": 0.014856618581688963, "context": 0.018145518530218396}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 3.206317663192749, "loss_scale": 4.0, "retries": 0, "seconds": 124.1, "max_memory_gb": 17.13, "ce": 0.6005472770775668, "kl": 0.010673774857423268, "context": 0.6058841901831329}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.9065015912055969, "loss_scale": 4.0, "retries": 0, "seconds": 125.8, "max_memory_gb": 17.13, "ce": 0.013252486911369488, "kl": 0.003266888531470613, "context": 0.014885931537719443}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.12968167662620544, "loss_scale": 4.0, "retries": 0, "seconds": 127.7, "max_memory_gb": 17.13, "ce": 0.006615103193325922, "kl": 0.0058745000351336785, "context": 0.009552352770697325}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.9143139123916626, "loss_scale": 4.0, "retries": 0, "seconds": 129.4, "max_memory_gb": 17.13, "ce": 0.01324550980643835, "kl": 0.4395955471554771, "context": 0.23304328898666427}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 1.9344207048416138, "loss_scale": 4.0, "retries": 0, "seconds": 131.3, "max_memory_gb": 17.13, "ce": 0.15750685936654918, "kl": 0.029494398972019553, "context": 0.17225405701901764}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.1719307899475098, "loss_scale": 4.0, "retries": 0, "seconds": 133.0, "max_memory_gb": 17.13, "ce": 0.09271349490154535, "kl": 0.09086661762739823, "context": 0.13814680369978305}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.922913908958435, "loss_scale": 4.0, "retries": 0, "seconds": 134.9, "max_memory_gb": 17.13, "ce": 0.5119532057433389, "kl": 0.7974287630058825, "context": 0.9106675873044878}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 1.7987823486328125, "loss_scale": 4.0, "retries": 0, "seconds": 136.6, "max_memory_gb": 17.13, "ce": 0.015332163791754283, "kl": 1.2286401118617505, "context": 0.6296522245975211}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.32813185453414917, "loss_scale": 4.0, "retries": 0, "seconds": 138.5, "max_memory_gb": 17.13, "ce": 0.014380253618583083, "kl": 0.013601503043901175, "context": 0.021181005286052823}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.6311110258102417, "loss_scale": 4.0, "retries": 0, "seconds": 140.2, "max_memory_gb": 17.13, "ce": 0.0023493920889450237, "kl": 0.11441552090400364, "context": 0.059557152329944074}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.4396727383136749, "loss_scale": 4.0, "retries": 0, "seconds": 142.0, "max_memory_gb": 17.13, "ce": 0.013496075102011673, "kl": 0.015122757637072937, "context": 0.021057454068795778}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.3992208242416382, "loss_scale": 4.0, "retries": 0, "seconds": 143.7, "max_memory_gb": 17.13, "ce": 0.0028072746645193547, "kl": 0.05111963634408312, "context": 0.028367093618726358}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 1.032424807548523, "loss_scale": 4.0, "retries": 0, "seconds": 145.6, "max_memory_gb": 17.13, "ce": 0.11076539871282876, "kl": 0.04296397613143199, "context": 0.13224738196004182}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.1937544345855713, "loss_scale": 4.0, "retries": 0, "seconds": 147.3, "max_memory_gb": 17.13, "ce": 0.014367772964760661, "kl": 0.004915162739052903, "context": 0.016825354658067226}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.9889070987701416, "loss_scale": 4.0, "retries": 0, "seconds": 149.1, "max_memory_gb": 17.13, "ce": 0.11479575114208274, "kl": 0.47938725063431775, "context": 0.3544893776124809}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 1.9069175720214844, "loss_scale": 4.0, "retries": 0, "seconds": 150.9, "max_memory_gb": 17.13, "ce": 0.15125561898457818, "kl": 0.029253687743221235, "context": 0.1658824700571131}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.1269819140434265, "loss_scale": 4.0, "retries": 0, "seconds": 152.6, "max_memory_gb": 17.13, "ce": 0.003828000306384638, "kl": 0.0008052148251067592, "context": 0.0042306077521061525}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 1.2868696451187134, "loss_scale": 4.0, "retries": 0, "seconds": 154.5, "max_memory_gb": 17.13, "ce": 0.03846834390424192, "kl": 0.1884308402077295, "context": 0.13268376397900283}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 1.7849841117858887, "loss_scale": 4.0, "retries": 0, "seconds": 156.3, "max_memory_gb": 17.13, "ce": 0.007311723777092993, "kl": 0.46345870691584423, "context": 0.23904108069837093}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.31861910223960876, "loss_scale": 4.0, "retries": 0, "seconds": 158.1, "max_memory_gb": 17.13, "ce": 0.01695101708173752, "kl": 0.0031537519251685353, "context": 0.018527892549172975}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.4321463108062744, "loss_scale": 4.0, "retries": 0, "seconds": 160.2, "max_memory_gb": 17.13, "ce": 0.02978999519109493, "kl": 0.01991689372516703, "context": 0.03974844289768953}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.1774301826953888, "loss_scale": 4.0, "retries": 0, "seconds": 161.9, "max_memory_gb": 17.13, "ce": 0.015338360302848741, "kl": 0.0042018108360935, "context": 0.017439265444409102}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 3.03821063041687, "loss_scale": 4.0, "retries": 0, "seconds": 163.6, "max_memory_gb": 17.13, "ce": 0.37057367419765797, "kl": 0.27804524697421584, "context": 0.5095962911727838}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 4.535595417022705, "loss_scale": 4.0, "retries": 0, "seconds": 165.5, "max_memory_gb": 17.13, "ce": 0.14255846649757586, "kl": 0.2251733867205985, "context": 0.2551451597828418}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 4.149247169494629, "loss_scale": 4.0, "retries": 0, "seconds": 167.2, "max_memory_gb": 17.13, "ce": 0.07293147014570422, "kl": 0.29610793814936187, "context": 0.2209854377724696}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 1.9474786520004272, "loss_scale": 4.0, "retries": 0, "seconds": 169.1, "max_memory_gb": 17.13, "ce": 0.06766400410560891, "kl": 0.025920217478414997, "context": 0.08062411478022113}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 3.6616883277893066, "loss_scale": 4.0, "retries": 0, "seconds": 170.8, "max_memory_gb": 17.13, "ce": 0.48274361540097743, "kl": 0.017707792459987104, "context": 0.4915975172771141}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 2.6359829902648926, "loss_scale": 4.0, "retries": 0, "seconds": 172.7, "max_memory_gb": 17.13, "ce": 0.3119730695107137, "kl": 0.006511924673759495, "context": 0.31522902022697963}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.4722032845020294, "loss_scale": 4.0, "retries": 0, "seconds": 174.4, "max_memory_gb": 17.13, "ce": 0.01620275160530582, "kl": 0.02028023195452988, "context": 0.026342867873609066}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.39967042207717896, "loss_scale": 4.0, "retries": 0, "seconds": 176.3, "max_memory_gb": 17.13, "ce": 0.002428730731480755, "kl": 0.08308163304172922, "context": 0.04396954878757242}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.15080416202545166, "loss_scale": 4.0, "retries": 0, "seconds": 177.9, "max_memory_gb": 17.13, "ce": 0.017192767438245937, "kl": 0.005193065508137806, "context": 0.019789300153206568}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.9063684940338135, "loss_scale": 4.0, "retries": 0, "seconds": 179.7, "max_memory_gb": 17.13, "ce": 0.040590692282421514, "kl": 0.3051822600646119, "context": 0.19318182347342372}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.21025267243385315, "loss_scale": 4.0, "retries": 0, "seconds": 181.4, "max_memory_gb": 17.13, "ce": 0.005838065058924258, "kl": 0.020464421046199277, "context": 0.016070275101810694}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.3718203902244568, "loss_scale": 4.0, "retries": 0, "seconds": 183.3, "max_memory_gb": 17.13, "ce": 0.031055286584887654, "kl": 0.010178757744142786, "context": 0.03614466590806842}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.2111429125070572, "loss_scale": 4.0, "retries": 0, "seconds": 185.0, "max_memory_gb": 17.13, "ce": 0.013251988988486119, "kl": 0.005506233119376702, "context": 0.016005105717340484}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.21858100593090057, "loss_scale": 4.0, "retries": 0, "seconds": 186.7, "max_memory_gb": 17.13, "ce": 0.012561596755404025, "kl": 0.003475692954907572, "context": 0.014299443311756477}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 0.9805336594581604, "loss_scale": 4.0, "retries": 0, "seconds": 188.5, "max_memory_gb": 17.13, "ce": 0.019832700403640047, "kl": 0.31945434733643197, "context": 0.17955987568711862}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.03436337411403656, "loss_scale": 4.0, "retries": 0, "seconds": 190.4, "max_memory_gb": 17.13, "ce": 0.0027320621156832203, "kl": 0.0004263911978341639, "context": 0.0029452577437041327}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.5740390419960022, "loss_scale": 4.0, "retries": 0, "seconds": 192.2, "max_memory_gb": 17.13, "ce": 0.04226633229700383, "kl": 0.13091808175067854, "context": 0.10772536938020494}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 2.4094622135162354, "loss_scale": 4.0, "retries": 0, "seconds": 194.3, "max_memory_gb": 17.13, "ce": 0.33938616880914196, "kl": 0.005796357012968656, "context": 0.3422843400912825}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 3.4592223167419434, "loss_scale": 4.0, "retries": 0, "seconds": 196.1, "max_memory_gb": 17.13, "ce": 0.23433290856337408, "kl": 0.13473149202764034, "context": 0.30169865768402815}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.5149887204170227, "loss_scale": 4.0, "retries": 0, "seconds": 198.0, "max_memory_gb": 17.13, "ce": 0.03659649472683668, "kl": 0.00358708668500185, "context": 0.03839003783650696}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.01892012730240822, "loss_scale": 4.0, "retries": 0, "seconds": 199.6, "max_memory_gb": 17.13, "ce": 0.001356776076136157, "kl": 0.0005029309431847651, "context": 0.0016082415531855077}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.3289010524749756, "loss_scale": 4.0, "retries": 0, "seconds": 201.4, "max_memory_gb": 17.13, "ce": 0.02171439555240795, "kl": 0.003821170197625179, "context": 0.02362498000729829}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 0.9939419031143188, "loss_scale": 4.0, "retries": 0, "seconds": 203.1, "max_memory_gb": 17.13, "ce": 0.010223072429653257, "kl": 0.08014550994266756, "context": 0.050295827561058104}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 2.3957679271698, "loss_scale": 4.0, "retries": 0, "seconds": 204.9, "max_memory_gb": 17.13, "ce": 0.016593004926107824, "kl": 0.36812933816918303, "context": 0.20065767048799898}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 1.372464656829834, "loss_scale": 4.0, "retries": 0, "seconds": 206.7, "max_memory_gb": 17.13, "ce": 0.029244155099149793, "kl": 0.09833769278338877, "context": 0.07841300149448216}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 1.0600320100784302, "loss_scale": 4.0, "retries": 0, "seconds": 208.4, "max_memory_gb": 17.13, "ce": 0.08973571678507142, "kl": 0.30377883645996917, "context": 0.24162513500778005}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.08970288187265396, "loss_scale": 4.0, "retries": 0, "seconds": 210.4, "max_memory_gb": 17.13, "ce": 0.004901318025076762, "kl": 0.0035104803173453547, "context": 0.006656558223767206}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.3621346950531006, "loss_scale": 4.0, "retries": 0, "seconds": 212.6, "max_memory_gb": 17.13, "ce": 0.23977782655856572, "kl": 0.0020014072506455705, "context": 0.24077853170456365}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.09879568219184875, "loss_scale": 4.0, "retries": 0, "seconds": 214.7, "max_memory_gb": 17.13, "ce": 0.004912416494335048, "kl": 0.0034858813983191794, "context": 0.0066553573415149}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 4.02910041809082, "loss_scale": 4.0, "retries": 0, "seconds": 216.3, "max_memory_gb": 17.13, "ce": 0.23062575078802183, "kl": 0.2080713885370642, "context": 0.33466144403791986}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.7957936525344849, "loss_scale": 4.0, "retries": 0, "seconds": 218.2, "max_memory_gb": 17.13, "ce": 0.056324183504329994, "kl": 0.03868279126982088, "context": 0.07566557959944475}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.3157860040664673, "loss_scale": 4.0, "retries": 0, "seconds": 219.9, "max_memory_gb": 17.13, "ce": 0.15307296617538668, "kl": 0.030726391418284038, "context": 0.16843616002006456}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.6366497278213501, "loss_scale": 4.0, "retries": 0, "seconds": 222.3, "max_memory_gb": 17.13, "ce": 0.04661520849913359, "kl": 0.02725078392541036, "context": 0.060240600258111954}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 3.164407968521118, "loss_scale": 4.0, "retries": 0, "seconds": 224.0, "max_memory_gb": 17.13, "ce": 0.6273988651955733, "kl": 0.4599276015533178, "context": 0.8573626836732728}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.9514873623847961, "loss_scale": 4.0, "retries": 0, "seconds": 225.9, "max_memory_gb": 17.13, "ce": 0.12834976080921479, "kl": 0.07816162106428237, "context": 0.16743057122221217}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 3.4863200187683105, "loss_scale": 4.0, "retries": 0, "seconds": 227.7, "max_memory_gb": 17.13, "ce": 0.6693057008087635, "kl": 0.09221510821953416, "context": 0.7154132565483451}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.10381209850311279, "loss_scale": 4.0, "retries": 0, "seconds": 229.6, "max_memory_gb": 17.13, "ce": 0.00645385542884469, "kl": 0.0006677474193566013, "context": 0.006787729304051027}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 2.7276177406311035, "loss_scale": 4.0, "retries": 0, "seconds": 231.3, "max_memory_gb": 17.13, "ce": 0.12486900779185817, "kl": 0.4762609809549758, "context": 0.3629994833609089}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 1.0321614742279053, "loss_scale": 4.0, "retries": 0, "seconds": 233.2, "max_memory_gb": 17.13, "ce": 0.09460303524974734, "kl": 0.03567140267841751, "context": 0.11243873645435087}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.5792543888092041, "loss_scale": 4.0, "retries": 0, "seconds": 234.9, "max_memory_gb": 17.13, "ce": 0.029071074328385293, "kl": 0.27040868054609746, "context": 0.1642754195490852}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 4.71050500869751, "loss_scale": 4.0, "retries": 0, "seconds": 236.8, "max_memory_gb": 17.13, "ce": 0.3083521784283221, "kl": 0.35560946688929107, "context": 0.48615691566374153}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 1.049155592918396, "loss_scale": 4.0, "retries": 0, "seconds": 238.5, "max_memory_gb": 17.13, "ce": 0.1381246635573916, "kl": 0.036671002799266716, "context": 0.1564601649879478}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 1.3362873792648315, "loss_scale": 4.0, "retries": 0, "seconds": 240.5, "max_memory_gb": 17.13, "ce": 0.3052267389575718, "kl": 0.021927048510406166, "context": 0.3161902692518197}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 1.0671637058258057, "loss_scale": 4.0, "retries": 0, "seconds": 242.3, "max_memory_gb": 17.13, "ce": 0.1862190684187226, "kl": 0.0036264032612507435, "context": 0.18803226421005093}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 0.4180726408958435, "loss_scale": 4.0, "retries": 0, "seconds": 244.4, "max_memory_gb": 17.13, "ce": 0.014331537822727114, "kl": 0.04396346807334339, "context": 0.03631327301263809}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 2.07415771484375, "loss_scale": 4.0, "retries": 0, "seconds": 246.1, "max_memory_gb": 17.13, "ce": 0.06200156745035201, "kl": 0.08662513868603128, "context": 0.10531414055731148}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 0.39659741520881653, "loss_scale": 4.0, "retries": 0, "seconds": 248.0, "max_memory_gb": 17.13, "ce": 0.015610487113008276, "kl": 0.020566347095154924, "context": 0.02589366065512877}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 2.727402448654175, "loss_scale": 4.0, "retries": 0, "seconds": 249.8, "max_memory_gb": 17.13, "ce": 0.09122522664256394, "kl": 0.13432894088327885, "context": 0.15838969312608242}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 1.1530712842941284, "loss_scale": 4.0, "retries": 0, "seconds": 251.7, "max_memory_gb": 17.13, "ce": 0.011982777316006832, "kl": 0.11411873022643704, "context": 0.06904214240785223}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.08988875895738602, "loss_scale": 4.0, "retries": 0, "seconds": 253.4, "max_memory_gb": 17.13, "ce": 0.006279426015680656, "kl": 0.0024149192195181968, "context": 0.007486885355319828}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.09460098296403885, "loss_scale": 4.0, "retries": 0, "seconds": 255.3, "max_memory_gb": 17.13, "ce": 0.009481302586209495, "kl": 0.00039386149001074955, "context": 0.009678233131126035}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 1.392229437828064, "loss_scale": 4.0, "retries": 0, "seconds": 257.0, "max_memory_gb": 17.13, "ce": 0.08125724201090634, "kl": 0.020361342089017853, "context": 0.0914379107998684}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 3.593001127243042, "loss_scale": 4.0, "retries": 0, "seconds": 258.9, "max_memory_gb": 17.13, "ce": 0.30865259616984986, "kl": 0.02436980550555745, "context": 0.32083749875891954}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.7034350633621216, "loss_scale": 4.0, "retries": 0, "seconds": 260.7, "max_memory_gb": 17.13, "ce": 0.029183412669226527, "kl": 0.10131772223394364, "context": 0.07984227378619835}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.02844235487282276, "loss_scale": 4.0, "retries": 0, "seconds": 262.4, "max_memory_gb": 17.13, "ce": 0.0016924760275287554, "kl": 0.0016801935980765847, "context": 0.0025325728638563305}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.19206546247005463, "loss_scale": 4.0, "retries": 0, "seconds": 264.2, "max_memory_gb": 17.13, "ce": 0.016416217826190405, "kl": 0.012751562710036524, "context": 0.022791998882894404}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.9441087245941162, "loss_scale": 4.0, "retries": 0, "seconds": 266.2, "max_memory_gb": 17.13, "ce": 0.3727672784589231, "kl": 0.3741024549017311, "context": 0.5598184952978045}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.236797496676445, "loss_scale": 4.0, "retries": 0, "seconds": 267.9, "max_memory_gb": 17.13, "ce": 0.012880601781944279, "kl": 0.00927053560008062, "context": 0.0175158696656581}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 0.20298780500888824, "loss_scale": 4.0, "retries": 0, "seconds": 269.8, "max_memory_gb": 17.13, "ce": 0.01734328764723614, "kl": 0.0015235644889344258, "context": 0.0181050700484775}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.5710281729698181, "loss_scale": 4.0, "retries": 0, "seconds": 271.5, "max_memory_gb": 17.13, "ce": 0.06629200877432595, "kl": 0.029861891533982998, "context": 0.0812229545917944}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.7115623950958252, "loss_scale": 4.0, "retries": 0, "seconds": 273.3, "max_memory_gb": 17.13, "ce": 0.024808512040181085, "kl": 0.026774618461786304, "context": 0.0381958203506656}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 0.7571110725402832, "loss_scale": 4.0, "retries": 0, "seconds": 275.1, "max_memory_gb": 17.13, "ce": 0.002126470790244639, "kl": 0.08088561206386657, "context": 0.0425692769640591}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.8573498725891113, "loss_scale": 4.0, "retries": 0, "seconds": 276.9, "max_memory_gb": 17.13, "ce": 0.00479956362687517, "kl": 0.4217919163147599, "context": 0.21569552224536892}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.48335304856300354, "loss_scale": 4.0, "retries": 0, "seconds": 278.8, "max_memory_gb": 17.13, "ce": 0.033716644036758225, "kl": 0.10217562573961914, "context": 0.08480446039902745}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 1.5779860019683838, "loss_scale": 4.0, "retries": 0, "seconds": 280.6, "max_memory_gb": 17.13, "ce": 0.24031467582972255, "kl": 0.012324016688580741, "context": 0.2464766776538454}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 1.0697426795959473, "loss_scale": 4.0, "retries": 0, "seconds": 282.6, "max_memory_gb": 17.13, "ce": 0.020821473663090728, "kl": 0.00579341708123593, "context": 0.02371818154642824}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.29826444387435913, "loss_scale": 4.0, "retries": 0, "seconds": 284.3, "max_memory_gb": 17.13, "ce": 0.021783442702144384, "kl": 0.028151228150818497, "context": 0.0358590567484498}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.4933348894119263, "loss_scale": 4.0, "retries": 0, "seconds": 286.2, "max_memory_gb": 17.13, "ce": 0.16958948416868225, "kl": 0.13156796038674656, "context": 0.23537346522789448}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.6925236582756042, "loss_scale": 4.0, "retries": 0, "seconds": 287.9, "max_memory_gb": 17.13, "ce": 0.006102523184381425, "kl": 0.039618896291358396, "context": 0.02591197134461254}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 1.2320799827575684, "loss_scale": 4.0, "retries": 0, "seconds": 289.8, "max_memory_gb": 17.13, "ce": 0.06242876726901159, "kl": 0.24802087138596107, "context": 0.18643920810427517}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 2.2124340534210205, "loss_scale": 4.0, "retries": 0, "seconds": 291.5, "max_memory_gb": 17.13, "ce": 0.18536058644531295, "kl": 0.15410461083865812, "context": 0.26241289931931533}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 1.918212890625, "loss_scale": 4.0, "retries": 0, "seconds": 293.4, "max_memory_gb": 17.13, "ce": 0.09504336790996604, "kl": 0.12616455401075655, "context": 0.1581256448989734}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 5.624154567718506, "loss_scale": 4.0, "retries": 0, "seconds": 295.2, "max_memory_gb": 17.13, "ce": 0.20937424783187453, "kl": 0.6516481204162119, "context": 0.5351983286382165}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 1.9518234729766846, "loss_scale": 4.0, "retries": 0, "seconds": 297.3, "max_memory_gb": 17.13, "ce": 0.08436312030971749, "kl": 0.04162357918721682, "context": 0.10517491112841526}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.17822125554084778, "loss_scale": 4.0, "retries": 0, "seconds": 299.0, "max_memory_gb": 17.13, "ce": 0.011971026222454384, "kl": 0.007746344148472417, "context": 0.015844198351260275}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 2.1030187606811523, "loss_scale": 4.0, "retries": 0, "seconds": 300.9, "max_memory_gb": 17.13, "ce": 0.35452935425564647, "kl": 0.04861018986412091, "context": 0.37883445364423096}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 1.9017494916915894, "loss_scale": 4.0, "retries": 0, "seconds": 302.6, "max_memory_gb": 17.13, "ce": 0.04099743512051646, "kl": 0.010284532872901764, "context": 0.04613970179343596}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.13451653718948364, "loss_scale": 4.0, "retries": 0, "seconds": 304.4, "max_memory_gb": 17.13, "ce": 0.010659021980245598, "kl": 0.0015414936351589859, "context": 0.011429768564994447}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 1.736571192741394, "loss_scale": 4.0, "retries": 0, "seconds": 306.1, "max_memory_gb": 17.13, "ce": 0.06514042482012883, "kl": 0.8945074211806059, "context": 0.512394146528095}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.01461161207407713, "loss_scale": 4.0, "retries": 0, "seconds": 308.0, "max_memory_gb": 17.13, "ce": 0.0010190612010774203, "kl": 6.542657547470299e-05, "context": 0.0010517744958633557}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.9263900518417358, "loss_scale": 4.0, "retries": 0, "seconds": 309.8, "max_memory_gb": 17.13, "ce": 0.017228182157850824, "kl": 0.04258425661828369, "context": 0.03852031024871394}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.1456766426563263, "loss_scale": 4.0, "retries": 0, "seconds": 311.7, "max_memory_gb": 17.13, "ce": 0.0074643924162955955, "kl": 0.010960732819512486, "context": 0.012944759408128448}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.1659599393606186, "loss_scale": 4.0, "retries": 0, "seconds": 313.4, "max_memory_gb": 17.13, "ce": 0.00965834892122075, "kl": 0.004157901217695326, "context": 0.011737299268133938}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.7239435911178589, "loss_scale": 4.0, "retries": 0, "seconds": 315.1, "max_memory_gb": 17.13, "ce": 0.04300318342575338, "kl": 0.006433557713535265, "context": 0.04621996321657207}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 1.1723825931549072, "loss_scale": 4.0, "retries": 0, "seconds": 316.9, "max_memory_gb": 17.13, "ce": 0.08474576764274389, "kl": 0.03072280646301806, "context": 0.10010717203840613}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.07275929301977158, "loss_scale": 4.0, "retries": 0, "seconds": 318.7, "max_memory_gb": 17.13, "ce": 0.004453691828530282, "kl": 0.0005437056570372079, "context": 0.004725544742541388}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.9562514424324036, "loss_scale": 4.0, "retries": 0, "seconds": 320.5, "max_memory_gb": 17.13, "ce": 0.05267592571181012, "kl": 0.6147136360214063, "context": 0.3600327474414371}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 1.9644443988800049, "loss_scale": 4.0, "retries": 0, "seconds": 322.2, "max_memory_gb": 17.13, "ce": 0.13981380546465516, "kl": 0.012509532418334857, "context": 0.14606857812032104}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.3289203643798828, "loss_scale": 4.0, "retries": 0, "seconds": 324.1, "max_memory_gb": 17.13, "ce": 0.34602531633572653, "kl": 0.010948446251859423, "context": 0.3514995436707977}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.6643137335777283, "loss_scale": 4.0, "retries": 0, "seconds": 325.8, "max_memory_gb": 17.13, "ce": 0.0380977050808724, "kl": 0.029365846661676187, "context": 0.05278062750585377}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 0.8452438116073608, "loss_scale": 4.0, "retries": 0, "seconds": 327.7, "max_memory_gb": 17.13, "ce": 0.002609491697512567, "kl": 0.13031860467162915, "context": 0.06776879244716838}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.6115556359291077, "loss_scale": 4.0, "retries": 0, "seconds": 329.4, "max_memory_gb": 17.13, "ce": 0.09486125590046868, "kl": 0.16109499134336147, "context": 0.17540874460246414}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.045449983328580856, "loss_scale": 4.0, "retries": 0, "seconds": 331.3, "max_memory_gb": 17.13, "ce": 0.003029002560651861, "kl": 0.00018295792324352078, "context": 0.003120481487712823}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.0816832184791565, "loss_scale": 4.0, "retries": 0, "seconds": 333.1, "max_memory_gb": 17.13, "ce": 0.0027030316123273224, "kl": 0.002952465154521633, "context": 0.004179264302365482}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.5513705611228943, "loss_scale": 4.0, "retries": 0, "seconds": 334.9, "max_memory_gb": 17.13, "ce": 0.006177038005262148, "kl": 0.11403114441509388, "context": 0.06319260974851204}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 4.949731349945068, "loss_scale": 4.0, "retries": 0, "seconds": 336.7, "max_memory_gb": 17.13, "ce": 0.09318183540017344, "kl": 0.1444210129302519, "context": 0.1653923419507919}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 0.9266740083694458, "loss_scale": 4.0, "retries": 0, "seconds": 338.5, "max_memory_gb": 17.13, "ce": 0.012563333970319945, "kl": 0.5930091182999604, "context": 0.3090678928856505}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.6649374961853027, "loss_scale": 4.0, "retries": 0, "seconds": 340.3, "max_memory_gb": 17.13, "ce": 0.19769532809732482, "kl": 0.007327311663175351, "context": 0.20135898661101237}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 3.5223536491394043, "loss_scale": 4.0, "retries": 0, "seconds": 342.2, "max_memory_gb": 17.13, "ce": 0.3430483833508333, "kl": 0.21358762577438029, "context": 0.44984218505123863}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.04731576889753342, "loss_scale": 4.0, "retries": 0, "seconds": 343.9, "max_memory_gb": 17.13, "ce": 0.001759465092618484, "kl": 0.002481906663888367, "context": 0.0030004183936398476}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 5.740854263305664, "loss_scale": 4.0, "retries": 0, "seconds": 345.7, "max_memory_gb": 17.13, "ce": 0.2050032950064633, "kl": 0.14503986645650002, "context": 0.27752322843298316}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.2012287378311157, "loss_scale": 4.0, "retries": 0, "seconds": 347.4, "max_memory_gb": 17.13, "ce": 0.3816464228439145, "kl": 0.028004135611809033, "context": 0.39564848341979086}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.04416802152991295, "loss_scale": 4.0, "retries": 0, "seconds": 349.3, "max_memory_gb": 17.13, "ce": 0.002533141683670692, "kl": 0.0009827044805206242, "context": 0.003024493853445165}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.7708225846290588, "loss_scale": 4.0, "retries": 0, "seconds": 351.0, "max_memory_gb": 17.13, "ce": 0.06846456718631089, "kl": 0.005782325739346561, "context": 0.07135572994593531}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.12659277021884918, "loss_scale": 4.0, "retries": 0, "seconds": 352.8, "max_memory_gb": 17.13, "ce": 0.013620834273751825, "kl": 0.000711613040039083, "context": 0.013976640824694186}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 1.0687675476074219, "loss_scale": 4.0, "retries": 0, "seconds": 354.6, "max_memory_gb": 17.13, "ce": 0.003652001927548554, "kl": 0.1743759700194687, "context": 0.0908399890322471}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 3.3905282020568848, "loss_scale": 4.0, "retries": 0, "seconds": 356.4, "max_memory_gb": 17.13, "ce": 0.039958897468750365, "kl": 0.4736772943560936, "context": 0.27679754979908466}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.43249332904815674, "loss_scale": 4.0, "retries": 0, "seconds": 358.2, "max_memory_gb": 17.13, "ce": 0.07605435473669786, "kl": 0.007434810964696226, "context": 0.07977176026906818}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.0593233108520508, "loss_scale": 4.0, "retries": 0, "seconds": 360.1, "max_memory_gb": 17.13, "ce": 0.08106973764370196, "kl": 0.005396653948992025, "context": 0.08376806526212022}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.2585173845291138, "loss_scale": 4.0, "retries": 0, "seconds": 361.8, "max_memory_gb": 17.13, "ce": 0.2390213324688375, "kl": 0.02475460191089951, "context": 0.2513986323028803}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 1.2963200807571411, "loss_scale": 4.0, "retries": 0, "seconds": 363.7, "max_memory_gb": 17.13, "ce": 0.07252630323637277, "kl": 0.08533100985732744, "context": 0.11519180805771612}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.052394766360521317, "loss_scale": 4.0, "retries": 0, "seconds": 365.4, "max_memory_gb": 17.13, "ce": 0.0029355690494412556, "kl": 0.002184361706213167, "context": 0.0040277498483192176}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.8541053533554077, "loss_scale": 4.0, "retries": 0, "seconds": 367.3, "max_memory_gb": 17.13, "ce": 0.026078369497554377, "kl": 0.0279392983939033, "context": 0.04004801892733667}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.36492103338241577, "loss_scale": 4.0, "retries": 0, "seconds": 369.1, "max_memory_gb": 17.13, "ce": 0.026373289118055254, "kl": 0.0031457135046366602, "context": 0.027946146321482956}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 1.2888530492782593, "loss_scale": 4.0, "retries": 0, "seconds": 370.9, "max_memory_gb": 17.13, "ce": 0.006673387455521151, "kl": 0.4031338739696366, "context": 0.20824032576638274}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.10091938078403473, "loss_scale": 4.0, "retries": 0, "seconds": 372.8, "max_memory_gb": 17.13, "ce": 0.005135822415468283, "kl": 0.0016966106704785489, "context": 0.0059841277688974515}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.5239372253417969, "loss_scale": 4.0, "retries": 0, "seconds": 374.6, "max_memory_gb": 17.13, "ce": 0.07950497861020267, "kl": 0.06096124125178903, "context": 0.1099855990614742}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.03941752389073372, "loss_scale": 4.0, "retries": 0, "seconds": 376.4, "max_memory_gb": 17.13, "ce": 0.002565518574556336, "kl": 0.0009450313227716833, "context": 0.003038034134078771}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 3.411807060241699, "loss_scale": 4.0, "retries": 0, "seconds": 378.2, "max_memory_gb": 17.13, "ce": 0.20033634165883996, "kl": 0.042771863722009584, "context": 0.22172228177078068}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 3.2655553817749023, "loss_scale": 4.0, "retries": 0, "seconds": 379.8, "max_memory_gb": 17.13, "ce": 0.4441823743254645, "kl": 0.10569790258159628, "context": 0.49703132576541975}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.6065430045127869, "loss_scale": 4.0, "retries": 0, "seconds": 381.3, "max_memory_gb": 17.13, "ce": 0.12270827745669521, "kl": 0.05525383464191691, "context": 0.15033519407734275}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 3.199769973754883, "loss_scale": 4.0, "retries": 0, "seconds": 382.8, "max_memory_gb": 17.13, "ce": 0.1046379835170228, "kl": 0.909250474669534, "context": 0.5592632279149257}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.8040504455566406, "loss_scale": 4.0, "retries": 0, "seconds": 384.6, "max_memory_gb": 17.13, "ce": 0.18533102844958194, "kl": 0.024319241201737896, "context": 0.1974906529940199}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.7530766725540161, "loss_scale": 4.0, "retries": 0, "seconds": 386.5, "max_memory_gb": 17.13, "ce": 0.3072142790624639, "kl": 0.06601786869578063, "context": 0.3402232172084041}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 2.418272018432617, "loss_scale": 4.0, "retries": 0, "seconds": 388.2, "max_memory_gb": 17.13, "ce": 0.47071371268248186, "kl": 0.018631388584253727, "context": 0.4800294013984967}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.7133504748344421, "loss_scale": 4.0, "retries": 0, "seconds": 390.1, "max_memory_gb": 17.13, "ce": 0.03428022004663944, "kl": 0.07326565525545448, "context": 0.07091304910136387}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.0589715838432312, "loss_scale": 4.0, "retries": 0, "seconds": 391.8, "max_memory_gb": 17.13, "ce": 0.0039126699266489595, "kl": 0.0017124062906077597, "context": 0.004768873128341511}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 1.3059695959091187, "loss_scale": 4.0, "retries": 0, "seconds": 393.7, "max_memory_gb": 17.13, "ce": 0.16789381444687024, "kl": 0.21884211897850037, "context": 0.27731487713754177}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 1.0666669607162476, "loss_scale": 4.0, "retries": 0, "seconds": 395.4, "max_memory_gb": 17.13, "ce": 0.011003364459611475, "kl": 0.21027841118484503, "context": 0.11614256596658379}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 0.24320384860038757, "loss_scale": 4.0, "retries": 0, "seconds": 397.3, "max_memory_gb": 17.13, "ce": 0.01775114968768321, "kl": 0.0013899808109272271, "context": 0.018446140020387247}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.5752902030944824, "loss_scale": 4.0, "retries": 0, "seconds": 399.0, "max_memory_gb": 17.13, "ce": 0.0031470190806430764, "kl": 0.0612553622595442, "context": 0.033774700423236936}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 1.3374969959259033, "loss_scale": 4.0, "retries": 0, "seconds": 400.9, "max_memory_gb": 17.13, "ce": 0.07902309924247675, "kl": 0.05292597653942721, "context": 0.10548609052784741}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 2.840214967727661, "loss_scale": 4.0, "retries": 0, "seconds": 402.6, "max_memory_gb": 17.13, "ce": 0.14492824132321402, "kl": 0.008030443277675658, "context": 0.14894346497021616}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 1.0718656778335571, "loss_scale": 4.0, "retries": 0, "seconds": 404.5, "max_memory_gb": 17.13, "ce": 0.04423819313524291, "kl": 0.045031317412394856, "context": 0.0667538518900983}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.16942812502384186, "loss_scale": 4.0, "retries": 0, "seconds": 406.2, "max_memory_gb": 17.13, "ce": 0.005311717701260932, "kl": 0.013794701095321216, "context": 0.012209068168886006}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.45434802770614624, "loss_scale": 4.0, "retries": 0, "seconds": 408.1, "max_memory_gb": 17.13, "ce": 0.02589208449353464, "kl": 0.005885296807718987, "context": 0.028834732467657886}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.4529893696308136, "loss_scale": 4.0, "retries": 0, "seconds": 409.8, "max_memory_gb": 17.13, "ce": 0.01851203823753167, "kl": 0.011058501673687715, "context": 0.024041288168518804}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.39744359254837036, "loss_scale": 4.0, "retries": 0, "seconds": 411.6, "max_memory_gb": 17.13, "ce": 0.00993852413375862, "kl": 0.04348701669368893, "context": 0.03168203146196902}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 2.6758360862731934, "loss_scale": 4.0, "retries": 0, "seconds": 413.4, "max_memory_gb": 17.13, "ce": 0.08255273006216157, "kl": 0.04067814458284147, "context": 0.10289180050313007}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 2.5609796047210693, "loss_scale": 4.0, "retries": 0, "seconds": 415.2, "max_memory_gb": 17.13, "ce": 0.4192151690949686, "kl": 0.28154886193806306, "context": 0.559989579487592}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.6835744380950928, "loss_scale": 4.0, "retries": 0, "seconds": 416.8, "max_memory_gb": 17.13, "ce": 0.04106007644440979, "kl": 0.06220235554064857, "context": 0.07216125135892071}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.3913823664188385, "loss_scale": 4.0, "retries": 0, "seconds": 418.6, "max_memory_gb": 17.13, "ce": 0.024285109277116135, "kl": 0.005013429250539048, "context": 0.026791823416715488}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 1.5167523622512817, "loss_scale": 4.0, "retries": 0, "seconds": 420.3, "max_memory_gb": 17.13, "ce": 0.02658589428756386, "kl": 0.11119202384725213, "context": 0.08218190574552864}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.5486159920692444, "loss_scale": 4.0, "retries": 0, "seconds": 421.9, "max_memory_gb": 17.13, "ce": 0.02583607644191943, "kl": 0.013234343744443322, "context": 0.03245324830641039}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 2.499887704849243, "loss_scale": 4.0, "retries": 0, "seconds": 423.5, "max_memory_gb": 17.13, "ce": 0.3151549302856438, "kl": 0.01505093323339679, "context": 0.3226804044097662}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.2109016329050064, "loss_scale": 4.0, "retries": 0, "seconds": 425.2, "max_memory_gb": 17.13, "ce": 0.01523140090284869, "kl": 0.017229667493666057, "context": 0.023846234660595655}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.7437591552734375, "loss_scale": 4.0, "retries": 0, "seconds": 427.0, "max_memory_gb": 17.13, "ce": 0.06461043225135654, "kl": 0.021637795233800716, "context": 0.07542933063814417}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.3352600932121277, "loss_scale": 4.0, "retries": 0, "seconds": 428.8, "max_memory_gb": 17.13, "ce": 0.020668179538915865, "kl": 0.004237631663272623, "context": 0.022786995803471655}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.4359774887561798, "loss_scale": 4.0, "retries": 0, "seconds": 430.6, "max_memory_gb": 17.13, "ce": 0.10827227460686117, "kl": 0.013662457808095496, "context": 0.11510350450407714}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 0.3227763772010803, "loss_scale": 4.0, "retries": 0, "seconds": 432.3, "max_memory_gb": 17.13, "ce": 0.018222975253593177, "kl": 0.02412616603760398, "context": 0.03028605825966224}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 1.3807390928268433, "loss_scale": 4.0, "retries": 0, "seconds": 433.9, "max_memory_gb": 17.13, "ce": 0.07964697590796277, "kl": 0.025736128649441525, "context": 0.09251503995619714}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.2091599851846695, "loss_scale": 4.0, "retries": 0, "seconds": 435.7, "max_memory_gb": 17.13, "ce": 0.012126156347221695, "kl": 0.014747380682820221, "context": 0.019499846210237592}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.059382129460573196, "loss_scale": 4.0, "retries": 0, "seconds": 437.6, "max_memory_gb": 17.13, "ce": 0.003704319038661197, "kl": 0.0012496678828028962, "context": 0.004329152958234772}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 2.926884651184082, "loss_scale": 4.0, "retries": 0, "seconds": 439.3, "max_memory_gb": 17.13, "ce": 0.02863648192578694, "kl": 0.2162682740499804, "context": 0.1367706189485034}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 1.7552579641342163, "loss_scale": 4.0, "retries": 0, "seconds": 441.3, "max_memory_gb": 17.13, "ce": 0.3041269403183833, "kl": 0.03307048497663345, "context": 0.3206621868885122}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.02985933981835842, "loss_scale": 4.0, "retries": 0, "seconds": 443.1, "max_memory_gb": 17.13, "ce": 0.001840578297560569, "kl": 0.0004146127830608748, "context": 0.0020478846709011123}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.9986809492111206, "loss_scale": 4.0, "retries": 0, "seconds": 445.1, "max_memory_gb": 17.13, "ce": 0.010180306970141828, "kl": 0.0963044756117597, "context": 0.05833254489698447}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.28900283575057983, "loss_scale": 4.0, "retries": 0, "seconds": 446.9, "max_memory_gb": 17.13, "ce": 0.018310792511329055, "kl": 0.006968414876610041, "context": 0.021795000415295362}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 1.3031882047653198, "loss_scale": 4.0, "retries": 0, "seconds": 448.8, "max_memory_gb": 17.13, "ce": 0.08764075161889195, "kl": 0.07211861503310502, "context": 0.12370005855336785}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.15983697772026062, "loss_scale": 4.0, "retries": 0, "seconds": 450.5, "max_memory_gb": 17.13, "ce": 0.00977184681687504, "kl": 0.006191881791892229, "context": 0.012867787416325882}

runs/fixed_internvl/context/Full_s42/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Full_s42/adapter/adapter_config.json

@@ -0,0 +1,556 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.0.mlp.down_proj": 32,
+    "model.language_model.layers.0.mlp.gate_proj": 32,
+    "model.language_model.layers.0.mlp.up_proj": 32,
+    "model.language_model.layers.0.self_attn.k_proj": 32,
+    "model.language_model.layers.0.self_attn.o_proj": 32,
+    "model.language_model.layers.0.self_attn.q_proj": 32,
+    "model.language_model.layers.0.self_attn.v_proj": 32,
+    "model.language_model.layers.1.mlp.down_proj": 32,
+    "model.language_model.layers.1.mlp.gate_proj": 32,
+    "model.language_model.layers.1.mlp.up_proj": 32,
+    "model.language_model.layers.1.self_attn.k_proj": 32,
+    "model.language_model.layers.1.self_attn.o_proj": 32,
+    "model.language_model.layers.1.self_attn.q_proj": 32,
+    "model.language_model.layers.1.self_attn.v_proj": 32,
+    "model.language_model.layers.10.mlp.down_proj": 32,
+    "model.language_model.layers.10.mlp.gate_proj": 32,
+    "model.language_model.layers.10.mlp.up_proj": 32,
+    "model.language_model.layers.10.self_attn.k_proj": 32,
+    "model.language_model.layers.10.self_attn.o_proj": 32,
+    "model.language_model.layers.10.self_attn.q_proj": 32,
+    "model.language_model.layers.10.self_attn.v_proj": 32,
+    "model.language_model.layers.11.mlp.down_proj": 32,
+    "model.language_model.layers.11.mlp.gate_proj": 32,
+    "model.language_model.layers.11.mlp.up_proj": 32,
+    "model.language_model.layers.11.self_attn.k_proj": 32,
+    "model.language_model.layers.11.self_attn.o_proj": 32,
+    "model.language_model.layers.11.self_attn.q_proj": 32,
+    "model.language_model.layers.11.self_attn.v_proj": 32,
+    "model.language_model.layers.12.mlp.down_proj": 32,
+    "model.language_model.layers.12.mlp.gate_proj": 32,
+    "model.language_model.layers.12.mlp.up_proj": 32,
+    "model.language_model.layers.12.self_attn.k_proj": 32,
+    "model.language_model.layers.12.self_attn.o_proj": 32,
+    "model.language_model.layers.12.self_attn.q_proj": 32,
+    "model.language_model.layers.12.self_attn.v_proj": 32,
+    "model.language_model.layers.13.mlp.down_proj": 32,
+    "model.language_model.layers.13.mlp.gate_proj": 32,
+    "model.language_model.layers.13.mlp.up_proj": 32,
+    "model.language_model.layers.13.self_attn.k_proj": 32,
+    "model.language_model.layers.13.self_attn.o_proj": 32,
+    "model.language_model.layers.13.self_attn.q_proj": 32,
+    "model.language_model.layers.13.self_attn.v_proj": 32,
+    "model.language_model.layers.14.mlp.down_proj": 32,
+    "model.language_model.layers.14.mlp.gate_proj": 32,
+    "model.language_model.layers.14.mlp.up_proj": 32,
+    "model.language_model.layers.14.self_attn.k_proj": 32,
+    "model.language_model.layers.14.self_attn.o_proj": 32,
+    "model.language_model.layers.14.self_attn.q_proj": 32,
+    "model.language_model.layers.14.self_attn.v_proj": 32,
+    "model.language_model.layers.15.mlp.down_proj": 32,
+    "model.language_model.layers.15.mlp.gate_proj": 32,
+    "model.language_model.layers.15.mlp.up_proj": 32,
+    "model.language_model.layers.15.self_attn.k_proj": 32,
+    "model.language_model.layers.15.self_attn.o_proj": 32,
+    "model.language_model.layers.15.self_attn.q_proj": 32,
+    "model.language_model.layers.15.self_attn.v_proj": 32,
+    "model.language_model.layers.16.mlp.down_proj": 32,
+    "model.language_model.layers.16.mlp.gate_proj": 32,
+    "model.language_model.layers.16.mlp.up_proj": 32,
+    "model.language_model.layers.16.self_attn.k_proj": 32,
+    "model.language_model.layers.16.self_attn.o_proj": 32,
+    "model.language_model.layers.16.self_attn.q_proj": 32,
+    "model.language_model.layers.16.self_attn.v_proj": 32,
+    "model.language_model.layers.17.mlp.down_proj": 32,
+    "model.language_model.layers.17.mlp.gate_proj": 32,
+    "model.language_model.layers.17.mlp.up_proj": 32,
+    "model.language_model.layers.17.self_attn.k_proj": 32,
+    "model.language_model.layers.17.self_attn.o_proj": 32,
+    "model.language_model.layers.17.self_attn.q_proj": 32,
+    "model.language_model.layers.17.self_attn.v_proj": 32,
+    "model.language_model.layers.18.mlp.down_proj": 32,
+    "model.language_model.layers.18.mlp.gate_proj": 32,
+    "model.language_model.layers.18.mlp.up_proj": 32,
+    "model.language_model.layers.18.self_attn.k_proj": 32,
+    "model.language_model.layers.18.self_attn.o_proj": 32,
+    "model.language_model.layers.18.self_attn.q_proj": 32,
+    "model.language_model.layers.18.self_attn.v_proj": 32,
+    "model.language_model.layers.19.mlp.down_proj": 32,
+    "model.language_model.layers.19.mlp.gate_proj": 32,
+    "model.language_model.layers.19.mlp.up_proj": 32,
+    "model.language_model.layers.19.self_attn.k_proj": 32,
+    "model.language_model.layers.19.self_attn.o_proj": 32,
+    "model.language_model.layers.19.self_attn.q_proj": 32,
+    "model.language_model.layers.19.self_attn.v_proj": 32,
+    "model.language_model.layers.2.mlp.down_proj": 32,
+    "model.language_model.layers.2.mlp.gate_proj": 32,
+    "model.language_model.layers.2.mlp.up_proj": 32,
+    "model.language_model.layers.2.self_attn.k_proj": 32,
+    "model.language_model.layers.2.self_attn.o_proj": 32,
+    "model.language_model.layers.2.self_attn.q_proj": 32,
+    "model.language_model.layers.2.self_attn.v_proj": 32,
+    "model.language_model.layers.20.mlp.down_proj": 32,
+    "model.language_model.layers.20.mlp.gate_proj": 32,
+    "model.language_model.layers.20.mlp.up_proj": 32,
+    "model.language_model.layers.20.self_attn.k_proj": 32,
+    "model.language_model.layers.20.self_attn.o_proj": 32,
+    "model.language_model.layers.20.self_attn.q_proj": 32,
+    "model.language_model.layers.20.self_attn.v_proj": 32,
+    "model.language_model.layers.21.mlp.down_proj": 32,
+    "model.language_model.layers.21.mlp.gate_proj": 32,
+    "model.language_model.layers.21.mlp.up_proj": 32,
+    "model.language_model.layers.21.self_attn.k_proj": 32,
+    "model.language_model.layers.21.self_attn.o_proj": 32,
+    "model.language_model.layers.21.self_attn.q_proj": 32,
+    "model.language_model.layers.21.self_attn.v_proj": 32,
+    "model.language_model.layers.22.mlp.down_proj": 32,
+    "model.language_model.layers.22.mlp.gate_proj": 32,
+    "model.language_model.layers.22.mlp.up_proj": 32,
+    "model.language_model.layers.22.self_attn.k_proj": 32,
+    "model.language_model.layers.22.self_attn.o_proj": 32,
+    "model.language_model.layers.22.self_attn.q_proj": 32,
+    "model.language_model.layers.22.self_attn.v_proj": 32,
+    "model.language_model.layers.23.mlp.down_proj": 32,
+    "model.language_model.layers.23.mlp.gate_proj": 32,
+    "model.language_model.layers.23.mlp.up_proj": 32,
+    "model.language_model.layers.23.self_attn.k_proj": 32,
+    "model.language_model.layers.23.self_attn.o_proj": 32,
+    "model.language_model.layers.23.self_attn.q_proj": 32,
+    "model.language_model.layers.23.self_attn.v_proj": 32,
+    "model.language_model.layers.24.mlp.down_proj": 32,
+    "model.language_model.layers.24.mlp.gate_proj": 32,
+    "model.language_model.layers.24.mlp.up_proj": 32,
+    "model.language_model.layers.24.self_attn.k_proj": 32,
+    "model.language_model.layers.24.self_attn.o_proj": 32,
+    "model.language_model.layers.24.self_attn.q_proj": 32,
+    "model.language_model.layers.24.self_attn.v_proj": 32,
+    "model.language_model.layers.25.mlp.down_proj": 32,
+    "model.language_model.layers.25.mlp.gate_proj": 32,
+    "model.language_model.layers.25.mlp.up_proj": 32,
+    "model.language_model.layers.25.self_attn.k_proj": 32,
+    "model.language_model.layers.25.self_attn.o_proj": 32,
+    "model.language_model.layers.25.self_attn.q_proj": 32,
+    "model.language_model.layers.25.self_attn.v_proj": 32,
+    "model.language_model.layers.26.mlp.down_proj": 32,
+    "model.language_model.layers.26.mlp.gate_proj": 32,
+    "model.language_model.layers.26.mlp.up_proj": 32,
+    "model.language_model.layers.26.self_attn.k_proj": 32,
+    "model.language_model.layers.26.self_attn.o_proj": 32,
+    "model.language_model.layers.26.self_attn.q_proj": 32,
+    "model.language_model.layers.26.self_attn.v_proj": 32,
+    "model.language_model.layers.27.mlp.down_proj": 32,
+    "model.language_model.layers.27.mlp.gate_proj": 32,
+    "model.language_model.layers.27.mlp.up_proj": 32,
+    "model.language_model.layers.27.self_attn.k_proj": 32,
+    "model.language_model.layers.27.self_attn.o_proj": 32,
+    "model.language_model.layers.27.self_attn.q_proj": 32,
+    "model.language_model.layers.27.self_attn.v_proj": 32,
+    "model.language_model.layers.28.mlp.down_proj": 32,
+    "model.language_model.layers.28.mlp.gate_proj": 32,
+    "model.language_model.layers.28.mlp.up_proj": 32,
+    "model.language_model.layers.28.self_attn.k_proj": 32,
+    "model.language_model.layers.28.self_attn.o_proj": 32,
+    "model.language_model.layers.28.self_attn.q_proj": 32,
+    "model.language_model.layers.28.self_attn.v_proj": 32,
+    "model.language_model.layers.29.mlp.down_proj": 32,
+    "model.language_model.layers.29.mlp.gate_proj": 32,
+    "model.language_model.layers.29.mlp.up_proj": 32,
+    "model.language_model.layers.29.self_attn.k_proj": 32,
+    "model.language_model.layers.29.self_attn.o_proj": 32,
+    "model.language_model.layers.29.self_attn.q_proj": 32,
+    "model.language_model.layers.29.self_attn.v_proj": 32,
+    "model.language_model.layers.3.mlp.down_proj": 32,
+    "model.language_model.layers.3.mlp.gate_proj": 32,
+    "model.language_model.layers.3.mlp.up_proj": 32,
+    "model.language_model.layers.3.self_attn.k_proj": 32,
+    "model.language_model.layers.3.self_attn.o_proj": 32,
+    "model.language_model.layers.3.self_attn.q_proj": 32,
+    "model.language_model.layers.3.self_attn.v_proj": 32,
+    "model.language_model.layers.30.mlp.down_proj": 32,
+    "model.language_model.layers.30.mlp.gate_proj": 32,
+    "model.language_model.layers.30.mlp.up_proj": 32,
+    "model.language_model.layers.30.self_attn.k_proj": 32,
+    "model.language_model.layers.30.self_attn.o_proj": 32,
+    "model.language_model.layers.30.self_attn.q_proj": 32,
+    "model.language_model.layers.30.self_attn.v_proj": 32,
+    "model.language_model.layers.31.mlp.down_proj": 32,
+    "model.language_model.layers.31.mlp.gate_proj": 32,
+    "model.language_model.layers.31.mlp.up_proj": 32,
+    "model.language_model.layers.31.self_attn.k_proj": 32,
+    "model.language_model.layers.31.self_attn.o_proj": 32,
+    "model.language_model.layers.31.self_attn.q_proj": 32,
+    "model.language_model.layers.31.self_attn.v_proj": 32,
+    "model.language_model.layers.32.mlp.down_proj": 32,
+    "model.language_model.layers.32.mlp.gate_proj": 32,
+    "model.language_model.layers.32.mlp.up_proj": 32,
+    "model.language_model.layers.32.self_attn.k_proj": 32,
+    "model.language_model.layers.32.self_attn.o_proj": 32,
+    "model.language_model.layers.32.self_attn.q_proj": 32,
+    "model.language_model.layers.32.self_attn.v_proj": 32,
+    "model.language_model.layers.33.mlp.down_proj": 32,
+    "model.language_model.layers.33.mlp.gate_proj": 32,
+    "model.language_model.layers.33.mlp.up_proj": 32,
+    "model.language_model.layers.33.self_attn.k_proj": 32,
+    "model.language_model.layers.33.self_attn.o_proj": 32,
+    "model.language_model.layers.33.self_attn.q_proj": 32,
+    "model.language_model.layers.33.self_attn.v_proj": 32,
+    "model.language_model.layers.34.mlp.down_proj": 32,
+    "model.language_model.layers.34.mlp.gate_proj": 32,
+    "model.language_model.layers.34.mlp.up_proj": 32,
+    "model.language_model.layers.34.self_attn.k_proj": 32,
+    "model.language_model.layers.34.self_attn.o_proj": 32,
+    "model.language_model.layers.34.self_attn.q_proj": 32,
+    "model.language_model.layers.34.self_attn.v_proj": 32,
+    "model.language_model.layers.35.mlp.down_proj": 32,
+    "model.language_model.layers.35.mlp.gate_proj": 32,
+    "model.language_model.layers.35.mlp.up_proj": 32,
+    "model.language_model.layers.35.self_attn.k_proj": 32,
+    "model.language_model.layers.35.self_attn.o_proj": 32,
+    "model.language_model.layers.35.self_attn.q_proj": 32,
+    "model.language_model.layers.35.self_attn.v_proj": 32,
+    "model.language_model.layers.4.mlp.down_proj": 32,
+    "model.language_model.layers.4.mlp.gate_proj": 32,
+    "model.language_model.layers.4.mlp.up_proj": 32,
+    "model.language_model.layers.4.self_attn.k_proj": 32,
+    "model.language_model.layers.4.self_attn.o_proj": 32,
+    "model.language_model.layers.4.self_attn.q_proj": 32,
+    "model.language_model.layers.4.self_attn.v_proj": 32,
+    "model.language_model.layers.5.mlp.down_proj": 32,
+    "model.language_model.layers.5.mlp.gate_proj": 32,
+    "model.language_model.layers.5.mlp.up_proj": 32,
+    "model.language_model.layers.5.self_attn.k_proj": 32,
+    "model.language_model.layers.5.self_attn.o_proj": 32,
+    "model.language_model.layers.5.self_attn.q_proj": 32,
+    "model.language_model.layers.5.self_attn.v_proj": 32,
+    "model.language_model.layers.6.mlp.down_proj": 32,
+    "model.language_model.layers.6.mlp.gate_proj": 32,
+    "model.language_model.layers.6.mlp.up_proj": 32,
+    "model.language_model.layers.6.self_attn.k_proj": 32,
+    "model.language_model.layers.6.self_attn.o_proj": 32,
+    "model.language_model.layers.6.self_attn.q_proj": 32,
+    "model.language_model.layers.6.self_attn.v_proj": 32,
+    "model.language_model.layers.7.mlp.down_proj": 32,
+    "model.language_model.layers.7.mlp.gate_proj": 32,
+    "model.language_model.layers.7.mlp.up_proj": 32,
+    "model.language_model.layers.7.self_attn.k_proj": 32,
+    "model.language_model.layers.7.self_attn.o_proj": 32,
+    "model.language_model.layers.7.self_attn.q_proj": 32,
+    "model.language_model.layers.7.self_attn.v_proj": 32,
+    "model.language_model.layers.8.mlp.down_proj": 32,
+    "model.language_model.layers.8.mlp.gate_proj": 32,
+    "model.language_model.layers.8.mlp.up_proj": 32,
+    "model.language_model.layers.8.self_attn.k_proj": 32,
+    "model.language_model.layers.8.self_attn.o_proj": 32,
+    "model.language_model.layers.8.self_attn.q_proj": 32,
+    "model.language_model.layers.8.self_attn.v_proj": 32,
+    "model.language_model.layers.9.mlp.down_proj": 32,
+    "model.language_model.layers.9.mlp.gate_proj": 32,
+    "model.language_model.layers.9.mlp.up_proj": 32,
+    "model.language_model.layers.9.self_attn.k_proj": 32,
+    "model.language_model.layers.9.self_attn.o_proj": 32,
+    "model.language_model.layers.9.self_attn.q_proj": 32,
+    "model.language_model.layers.9.self_attn.v_proj": 32
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.0.mlp.down_proj": 16,
+    "model.language_model.layers.0.mlp.gate_proj": 16,
+    "model.language_model.layers.0.mlp.up_proj": 16,
+    "model.language_model.layers.0.self_attn.k_proj": 16,
+    "model.language_model.layers.0.self_attn.o_proj": 16,
+    "model.language_model.layers.0.self_attn.q_proj": 16,
+    "model.language_model.layers.0.self_attn.v_proj": 16,
+    "model.language_model.layers.1.mlp.down_proj": 16,
+    "model.language_model.layers.1.mlp.gate_proj": 16,
+    "model.language_model.layers.1.mlp.up_proj": 16,
+    "model.language_model.layers.1.self_attn.k_proj": 16,
+    "model.language_model.layers.1.self_attn.o_proj": 16,
+    "model.language_model.layers.1.self_attn.q_proj": 16,
+    "model.language_model.layers.1.self_attn.v_proj": 16,
+    "model.language_model.layers.10.mlp.down_proj": 16,
+    "model.language_model.layers.10.mlp.gate_proj": 16,
+    "model.language_model.layers.10.mlp.up_proj": 16,
+    "model.language_model.layers.10.self_attn.k_proj": 16,
+    "model.language_model.layers.10.self_attn.o_proj": 16,
+    "model.language_model.layers.10.self_attn.q_proj": 16,
+    "model.language_model.layers.10.self_attn.v_proj": 16,
+    "model.language_model.layers.11.mlp.down_proj": 16,
+    "model.language_model.layers.11.mlp.gate_proj": 16,
+    "model.language_model.layers.11.mlp.up_proj": 16,
+    "model.language_model.layers.11.self_attn.k_proj": 16,
+    "model.language_model.layers.11.self_attn.o_proj": 16,
+    "model.language_model.layers.11.self_attn.q_proj": 16,
+    "model.language_model.layers.11.self_attn.v_proj": 16,
+    "model.language_model.layers.12.mlp.down_proj": 16,
+    "model.language_model.layers.12.mlp.gate_proj": 16,
+    "model.language_model.layers.12.mlp.up_proj": 16,
+    "model.language_model.layers.12.self_attn.k_proj": 16,
+    "model.language_model.layers.12.self_attn.o_proj": 16,
+    "model.language_model.layers.12.self_attn.q_proj": 16,
+    "model.language_model.layers.12.self_attn.v_proj": 16,
+    "model.language_model.layers.13.mlp.down_proj": 16,
+    "model.language_model.layers.13.mlp.gate_proj": 16,
+    "model.language_model.layers.13.mlp.up_proj": 16,
+    "model.language_model.layers.13.self_attn.k_proj": 16,
+    "model.language_model.layers.13.self_attn.o_proj": 16,
+    "model.language_model.layers.13.self_attn.q_proj": 16,
+    "model.language_model.layers.13.self_attn.v_proj": 16,
+    "model.language_model.layers.14.mlp.down_proj": 16,
+    "model.language_model.layers.14.mlp.gate_proj": 16,
+    "model.language_model.layers.14.mlp.up_proj": 16,
+    "model.language_model.layers.14.self_attn.k_proj": 16,
+    "model.language_model.layers.14.self_attn.o_proj": 16,
+    "model.language_model.layers.14.self_attn.q_proj": 16,
+    "model.language_model.layers.14.self_attn.v_proj": 16,
+    "model.language_model.layers.15.mlp.down_proj": 16,
+    "model.language_model.layers.15.mlp.gate_proj": 16,
+    "model.language_model.layers.15.mlp.up_proj": 16,
+    "model.language_model.layers.15.self_attn.k_proj": 16,
+    "model.language_model.layers.15.self_attn.o_proj": 16,
+    "model.language_model.layers.15.self_attn.q_proj": 16,
+    "model.language_model.layers.15.self_attn.v_proj": 16,
+    "model.language_model.layers.16.mlp.down_proj": 16,
+    "model.language_model.layers.16.mlp.gate_proj": 16,
+    "model.language_model.layers.16.mlp.up_proj": 16,
+    "model.language_model.layers.16.self_attn.k_proj": 16,
+    "model.language_model.layers.16.self_attn.o_proj": 16,
+    "model.language_model.layers.16.self_attn.q_proj": 16,
+    "model.language_model.layers.16.self_attn.v_proj": 16,
+    "model.language_model.layers.17.mlp.down_proj": 16,
+    "model.language_model.layers.17.mlp.gate_proj": 16,
+    "model.language_model.layers.17.mlp.up_proj": 16,
+    "model.language_model.layers.17.self_attn.k_proj": 16,
+    "model.language_model.layers.17.self_attn.o_proj": 16,
+    "model.language_model.layers.17.self_attn.q_proj": 16,
+    "model.language_model.layers.17.self_attn.v_proj": 16,
+    "model.language_model.layers.18.mlp.down_proj": 16,
+    "model.language_model.layers.18.mlp.gate_proj": 16,
+    "model.language_model.layers.18.mlp.up_proj": 16,
+    "model.language_model.layers.18.self_attn.k_proj": 16,
+    "model.language_model.layers.18.self_attn.o_proj": 16,
+    "model.language_model.layers.18.self_attn.q_proj": 16,
+    "model.language_model.layers.18.self_attn.v_proj": 16,
+    "model.language_model.layers.19.mlp.down_proj": 16,
+    "model.language_model.layers.19.mlp.gate_proj": 16,
+    "model.language_model.layers.19.mlp.up_proj": 16,
+    "model.language_model.layers.19.self_attn.k_proj": 16,
+    "model.language_model.layers.19.self_attn.o_proj": 16,
+    "model.language_model.layers.19.self_attn.q_proj": 16,
+    "model.language_model.layers.19.self_attn.v_proj": 16,
+    "model.language_model.layers.2.mlp.down_proj": 16,
+    "model.language_model.layers.2.mlp.gate_proj": 16,
+    "model.language_model.layers.2.mlp.up_proj": 16,
+    "model.language_model.layers.2.self_attn.k_proj": 16,
+    "model.language_model.layers.2.self_attn.o_proj": 16,
+    "model.language_model.layers.2.self_attn.q_proj": 16,
+    "model.language_model.layers.2.self_attn.v_proj": 16,
+    "model.language_model.layers.20.mlp.down_proj": 16,
+    "model.language_model.layers.20.mlp.gate_proj": 16,
+    "model.language_model.layers.20.mlp.up_proj": 16,
+    "model.language_model.layers.20.self_attn.k_proj": 16,
+    "model.language_model.layers.20.self_attn.o_proj": 16,
+    "model.language_model.layers.20.self_attn.q_proj": 16,
+    "model.language_model.layers.20.self_attn.v_proj": 16,
+    "model.language_model.layers.21.mlp.down_proj": 16,
+    "model.language_model.layers.21.mlp.gate_proj": 16,
+    "model.language_model.layers.21.mlp.up_proj": 16,
+    "model.language_model.layers.21.self_attn.k_proj": 16,
+    "model.language_model.layers.21.self_attn.o_proj": 16,
+    "model.language_model.layers.21.self_attn.q_proj": 16,
+    "model.language_model.layers.21.self_attn.v_proj": 16,
+    "model.language_model.layers.22.mlp.down_proj": 16,
+    "model.language_model.layers.22.mlp.gate_proj": 16,
+    "model.language_model.layers.22.mlp.up_proj": 16,
+    "model.language_model.layers.22.self_attn.k_proj": 16,
+    "model.language_model.layers.22.self_attn.o_proj": 16,
+    "model.language_model.layers.22.self_attn.q_proj": 16,
+    "model.language_model.layers.22.self_attn.v_proj": 16,
+    "model.language_model.layers.23.mlp.down_proj": 16,
+    "model.language_model.layers.23.mlp.gate_proj": 16,
+    "model.language_model.layers.23.mlp.up_proj": 16,
+    "model.language_model.layers.23.self_attn.k_proj": 16,
+    "model.language_model.layers.23.self_attn.o_proj": 16,
+    "model.language_model.layers.23.self_attn.q_proj": 16,
+    "model.language_model.layers.23.self_attn.v_proj": 16,
+    "model.language_model.layers.24.mlp.down_proj": 16,
+    "model.language_model.layers.24.mlp.gate_proj": 16,
+    "model.language_model.layers.24.mlp.up_proj": 16,
+    "model.language_model.layers.24.self_attn.k_proj": 16,
+    "model.language_model.layers.24.self_attn.o_proj": 16,
+    "model.language_model.layers.24.self_attn.q_proj": 16,
+    "model.language_model.layers.24.self_attn.v_proj": 16,
+    "model.language_model.layers.25.mlp.down_proj": 16,
+    "model.language_model.layers.25.mlp.gate_proj": 16,
+    "model.language_model.layers.25.mlp.up_proj": 16,
+    "model.language_model.layers.25.self_attn.k_proj": 16,
+    "model.language_model.layers.25.self_attn.o_proj": 16,
+    "model.language_model.layers.25.self_attn.q_proj": 16,
+    "model.language_model.layers.25.self_attn.v_proj": 16,
+    "model.language_model.layers.26.mlp.down_proj": 16,
+    "model.language_model.layers.26.mlp.gate_proj": 16,
+    "model.language_model.layers.26.mlp.up_proj": 16,
+    "model.language_model.layers.26.self_attn.k_proj": 16,
+    "model.language_model.layers.26.self_attn.o_proj": 16,
+    "model.language_model.layers.26.self_attn.q_proj": 16,
+    "model.language_model.layers.26.self_attn.v_proj": 16,
+    "model.language_model.layers.27.mlp.down_proj": 16,
+    "model.language_model.layers.27.mlp.gate_proj": 16,
+    "model.language_model.layers.27.mlp.up_proj": 16,
+    "model.language_model.layers.27.self_attn.k_proj": 16,
+    "model.language_model.layers.27.self_attn.o_proj": 16,
+    "model.language_model.layers.27.self_attn.q_proj": 16,
+    "model.language_model.layers.27.self_attn.v_proj": 16,
+    "model.language_model.layers.28.mlp.down_proj": 16,
+    "model.language_model.layers.28.mlp.gate_proj": 16,
+    "model.language_model.layers.28.mlp.up_proj": 16,
+    "model.language_model.layers.28.self_attn.k_proj": 16,
+    "model.language_model.layers.28.self_attn.o_proj": 16,
+    "model.language_model.layers.28.self_attn.q_proj": 16,
+    "model.language_model.layers.28.self_attn.v_proj": 16,
+    "model.language_model.layers.29.mlp.down_proj": 16,
+    "model.language_model.layers.29.mlp.gate_proj": 16,
+    "model.language_model.layers.29.mlp.up_proj": 16,
+    "model.language_model.layers.29.self_attn.k_proj": 16,
+    "model.language_model.layers.29.self_attn.o_proj": 16,
+    "model.language_model.layers.29.self_attn.q_proj": 16,
+    "model.language_model.layers.29.self_attn.v_proj": 16,
+    "model.language_model.layers.3.mlp.down_proj": 16,
+    "model.language_model.layers.3.mlp.gate_proj": 16,
+    "model.language_model.layers.3.mlp.up_proj": 16,
+    "model.language_model.layers.3.self_attn.k_proj": 16,
+    "model.language_model.layers.3.self_attn.o_proj": 16,
+    "model.language_model.layers.3.self_attn.q_proj": 16,
+    "model.language_model.layers.3.self_attn.v_proj": 16,
+    "model.language_model.layers.30.mlp.down_proj": 16,
+    "model.language_model.layers.30.mlp.gate_proj": 16,
+    "model.language_model.layers.30.mlp.up_proj": 16,
+    "model.language_model.layers.30.self_attn.k_proj": 16,
+    "model.language_model.layers.30.self_attn.o_proj": 16,
+    "model.language_model.layers.30.self_attn.q_proj": 16,
+    "model.language_model.layers.30.self_attn.v_proj": 16,
+    "model.language_model.layers.31.mlp.down_proj": 16,
+    "model.language_model.layers.31.mlp.gate_proj": 16,
+    "model.language_model.layers.31.mlp.up_proj": 16,
+    "model.language_model.layers.31.self_attn.k_proj": 16,
+    "model.language_model.layers.31.self_attn.o_proj": 16,
+    "model.language_model.layers.31.self_attn.q_proj": 16,
+    "model.language_model.layers.31.self_attn.v_proj": 16,
+    "model.language_model.layers.32.mlp.down_proj": 16,
+    "model.language_model.layers.32.mlp.gate_proj": 16,
+    "model.language_model.layers.32.mlp.up_proj": 16,
+    "model.language_model.layers.32.self_attn.k_proj": 16,
+    "model.language_model.layers.32.self_attn.o_proj": 16,
+    "model.language_model.layers.32.self_attn.q_proj": 16,
+    "model.language_model.layers.32.self_attn.v_proj": 16,
+    "model.language_model.layers.33.mlp.down_proj": 16,
+    "model.language_model.layers.33.mlp.gate_proj": 16,
+    "model.language_model.layers.33.mlp.up_proj": 16,
+    "model.language_model.layers.33.self_attn.k_proj": 16,
+    "model.language_model.layers.33.self_attn.o_proj": 16,
+    "model.language_model.layers.33.self_attn.q_proj": 16,
+    "model.language_model.layers.33.self_attn.v_proj": 16,
+    "model.language_model.layers.34.mlp.down_proj": 16,
+    "model.language_model.layers.34.mlp.gate_proj": 16,
+    "model.language_model.layers.34.mlp.up_proj": 16,
+    "model.language_model.layers.34.self_attn.k_proj": 16,
+    "model.language_model.layers.34.self_attn.o_proj": 16,
+    "model.language_model.layers.34.self_attn.q_proj": 16,
+    "model.language_model.layers.34.self_attn.v_proj": 16,
+    "model.language_model.layers.35.mlp.down_proj": 16,
+    "model.language_model.layers.35.mlp.gate_proj": 16,
+    "model.language_model.layers.35.mlp.up_proj": 16,
+    "model.language_model.layers.35.self_attn.k_proj": 16,
+    "model.language_model.layers.35.self_attn.o_proj": 16,
+    "model.language_model.layers.35.self_attn.q_proj": 16,
+    "model.language_model.layers.35.self_attn.v_proj": 16,
+    "model.language_model.layers.4.mlp.down_proj": 16,
+    "model.language_model.layers.4.mlp.gate_proj": 16,
+    "model.language_model.layers.4.mlp.up_proj": 16,
+    "model.language_model.layers.4.self_attn.k_proj": 16,
+    "model.language_model.layers.4.self_attn.o_proj": 16,
+    "model.language_model.layers.4.self_attn.q_proj": 16,
+    "model.language_model.layers.4.self_attn.v_proj": 16,
+    "model.language_model.layers.5.mlp.down_proj": 16,
+    "model.language_model.layers.5.mlp.gate_proj": 16,
+    "model.language_model.layers.5.mlp.up_proj": 16,
+    "model.language_model.layers.5.self_attn.k_proj": 16,
+    "model.language_model.layers.5.self_attn.o_proj": 16,
+    "model.language_model.layers.5.self_attn.q_proj": 16,
+    "model.language_model.layers.5.self_attn.v_proj": 16,
+    "model.language_model.layers.6.mlp.down_proj": 16,
+    "model.language_model.layers.6.mlp.gate_proj": 16,
+    "model.language_model.layers.6.mlp.up_proj": 16,
+    "model.language_model.layers.6.self_attn.k_proj": 16,
+    "model.language_model.layers.6.self_attn.o_proj": 16,
+    "model.language_model.layers.6.self_attn.q_proj": 16,
+    "model.language_model.layers.6.self_attn.v_proj": 16,
+    "model.language_model.layers.7.mlp.down_proj": 16,
+    "model.language_model.layers.7.mlp.gate_proj": 16,
+    "model.language_model.layers.7.mlp.up_proj": 16,
+    "model.language_model.layers.7.self_attn.k_proj": 16,
+    "model.language_model.layers.7.self_attn.o_proj": 16,
+    "model.language_model.layers.7.self_attn.q_proj": 16,
+    "model.language_model.layers.7.self_attn.v_proj": 16,
+    "model.language_model.layers.8.mlp.down_proj": 16,
+    "model.language_model.layers.8.mlp.gate_proj": 16,
+    "model.language_model.layers.8.mlp.up_proj": 16,
+    "model.language_model.layers.8.self_attn.k_proj": 16,
+    "model.language_model.layers.8.self_attn.o_proj": 16,
+    "model.language_model.layers.8.self_attn.q_proj": 16,
+    "model.language_model.layers.8.self_attn.v_proj": 16,
+    "model.language_model.layers.9.mlp.down_proj": 16,
+    "model.language_model.layers.9.mlp.gate_proj": 16,
+    "model.language_model.layers.9.mlp.up_proj": 16,
+    "model.language_model.layers.9.self_attn.k_proj": 16,
+    "model.language_model.layers.9.self_attn.o_proj": 16,
+    "model.language_model.layers.9.self_attn.q_proj": 16,
+    "model.language_model.layers.9.self_attn.v_proj": 16
+  },
+  "revision": null,
+  "target_modules": [
+    "self_attn.k_proj",
+    "down_proj",
+    "up_proj",
+    "o_proj",
+    "gate_proj",
+    "self_attn.q_proj",
+    "self_attn.v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Full_s42/config.json

@@ -0,0 +1,99 @@
+{
+  "behavior": "context",
+  "seed": 42,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Full",
+    "layers": [
+      0,
+      1,
+      2,
+      3,
+      4,
+      5,
+      6,
+      7,
+      8,
+      9,
+      10,
+      11,
+      12,
+      13,
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21,
+      22,
+      23,
+      24,
+      25,
+      26,
+      27,
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35
+    ],
+    "ranks": {
+      "0": 16,
+      "1": 16,
+      "2": 16,
+      "3": 16,
+      "4": 16,
+      "5": 16,
+      "6": 16,
+      "7": 16,
+      "8": 16,
+      "9": 16,
+      "10": 16,
+      "11": 16,
+      "12": 16,
+      "13": 16,
+      "14": 16,
+      "15": 16,
+      "16": 16,
+      "17": 16,
+      "18": 16,
+      "19": 16,
+      "20": 16,
+      "21": 16,
+      "22": 16,
+      "23": 16,
+      "24": 16,
+      "25": 16,
+      "26": 16,
+      "27": 16,
+      "28": 16,
+      "29": 16,
+      "30": 16,
+      "31": 16,
+      "32": 16,
+      "33": 16,
+      "34": 16,
+      "35": 16
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Full_s42/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 12.182524681091309, "loss_scale": 4.0, "retries": 0, "seconds": 2.4, "max_memory_gb": 16.8, "ce": 0.6558216065168381, "kl": 0.11560958076734096, "context": 0.7136263996362686}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 7.330140113830566, "loss_scale": 4.0, "retries": 0, "seconds": 3.6, "max_memory_gb": 17.12, "ce": 0.6035067513578554, "kl": 0.06002547080953491, "context": 0.6335194694167399}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 14.783873558044434, "loss_scale": 4.0, "retries": 0, "seconds": 5.0, "max_memory_gb": 17.12, "ce": 0.6623954616952688, "kl": 0.03491578088141978, "context": 0.679853348992765}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 13.136568069458008, "loss_scale": 4.0, "retries": 0, "seconds": 6.2, "max_memory_gb": 17.12, "ce": 0.5367100651055807, "kl": 0.6834764908999205, "context": 0.8784482785267755}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 5.290089130401611, "loss_scale": 4.0, "retries": 0, "seconds": 7.7, "max_memory_gb": 17.12, "ce": 0.2612495964858681, "kl": 1.1763496645726264, "context": 0.8494244343601167}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 7.630552291870117, "loss_scale": 4.0, "retries": 0, "seconds": 9.1, "max_memory_gb": 17.12, "ce": 0.20547310332767665, "kl": 0.33684401819482446, "context": 0.3738951049745083}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 2.5793769359588623, "loss_scale": 4.0, "retries": 0, "seconds": 10.6, "max_memory_gb": 17.12, "ce": 0.15961230918765068, "kl": 0.33669913600897416, "context": 0.32796187326312065}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 2.216149091720581, "loss_scale": 4.0, "retries": 0, "seconds": 12.1, "max_memory_gb": 17.12, "ce": 0.21333406154872137, "kl": 0.090103216577063, "context": 0.2583856611063311}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 3.9425292015075684, "loss_scale": 4.0, "retries": 0, "seconds": 13.6, "max_memory_gb": 17.12, "ce": 0.5543693801155314, "kl": 0.11190481926314533, "context": 0.6103217841591686}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 7.33359956741333, "loss_scale": 4.0, "retries": 0, "seconds": 15.0, "max_memory_gb": 17.12, "ce": 0.06425004024640657, "kl": 0.29435441899113357, "context": 0.21142724459059536}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.7246609330177307, "loss_scale": 4.0, "retries": 0, "seconds": 16.5, "max_memory_gb": 17.12, "ce": 0.001843360821112583, "kl": 0.24457873543764208, "context": 0.12413273087167909}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 7.1579999923706055, "loss_scale": 4.0, "retries": 0, "seconds": 18.0, "max_memory_gb": 17.12, "ce": 0.40179447492118925, "kl": 1.0611763037741184, "context": 0.9323826308827847}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.750418484210968, "loss_scale": 4.0, "retries": 0, "seconds": 19.5, "max_memory_gb": 17.12, "ce": 0.0523758263734635, "kl": 0.04554174665827304, "context": 0.07514670048840344}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 0.6139821410179138, "loss_scale": 4.0, "retries": 0, "seconds": 20.9, "max_memory_gb": 17.12, "ce": 0.02139446252840571, "kl": 0.10470971104223281, "context": 0.07374931697268039}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 6.945537567138672, "loss_scale": 4.0, "retries": 0, "seconds": 22.5, "max_memory_gb": 17.12, "ce": 0.3977182940579951, "kl": 0.022993048311036546, "context": 0.40921481885015965}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 2.664193630218506, "loss_scale": 4.0, "retries": 0, "seconds": 23.9, "max_memory_gb": 17.12, "ce": 0.20510142665239073, "kl": 0.018985024628818792, "context": 0.21459393506665947}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 0.6821981072425842, "loss_scale": 4.0, "retries": 0, "seconds": 25.3, "max_memory_gb": 17.12, "ce": 0.005235165299382061, "kl": 0.0812718335801037, "context": 0.04587108053965494}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.8440549969673157, "loss_scale": 4.0, "retries": 0, "seconds": 26.5, "max_memory_gb": 17.12, "ce": 0.050183832296170294, "kl": 0.05291749304160476, "context": 0.07664258277509362}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.2476172000169754, "loss_scale": 4.0, "retries": 0, "seconds": 27.9, "max_memory_gb": 17.12, "ce": 0.005267592172458535, "kl": 0.01890145735524129, "context": 0.014718321264808765}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.42538538575172424, "loss_scale": 4.0, "retries": 0, "seconds": 29.2, "max_memory_gb": 17.12, "ce": 0.003436322251218371, "kl": 0.07619959691419353, "context": 0.04153612254594918}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.40389567613601685, "loss_scale": 4.0, "retries": 0, "seconds": 30.5, "max_memory_gb": 17.12, "ce": 0.0036606046051019803, "kl": 0.027814239106191962, "context": 0.017567724687978625}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.35870257019996643, "loss_scale": 4.0, "retries": 0, "seconds": 31.8, "max_memory_gb": 17.12, "ce": 0.018044657279460807, "kl": 0.009315358040112187, "context": 0.02270233589297277}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 3.495340347290039, "loss_scale": 4.0, "retries": 0, "seconds": 33.2, "max_memory_gb": 17.12, "ce": 0.07671864553412888, "kl": 0.4226874455716825, "context": 0.28806236550735775}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 1.7915996313095093, "loss_scale": 4.0, "retries": 0, "seconds": 34.4, "max_memory_gb": 17.12, "ce": 0.05386551972333109, "kl": 0.7774854278741259, "context": 0.44260824701632373}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 1.7127927541732788, "loss_scale": 4.0, "retries": 0, "seconds": 35.7, "max_memory_gb": 17.12, "ce": 0.11026345296704676, "kl": 0.19385667037204257, "context": 0.20719179563457146}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 2.173466444015503, "loss_scale": 4.0, "retries": 0, "seconds": 37.1, "max_memory_gb": 17.12, "ce": 0.07603724598357076, "kl": 0.0002578565544126832, "context": 0.0761661725682643}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 9.896637916564941, "loss_scale": 4.0, "retries": 0, "seconds": 38.6, "max_memory_gb": 17.12, "ce": 0.8826032146753278, "kl": 0.01397590145097638, "context": 0.8895912051666528}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 2.184372663497925, "loss_scale": 4.0, "retries": 0, "seconds": 39.9, "max_memory_gb": 17.12, "ce": 0.008606113318819553, "kl": 0.9021823837829288, "context": 0.4596972941653803}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 1.607622504234314, "loss_scale": 4.0, "retries": 0, "seconds": 41.4, "max_memory_gb": 17.12, "ce": 0.12326010726974346, "kl": 0.035148697294062003, "context": 0.14083445590222254}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 1.9135663509368896, "loss_scale": 4.0, "retries": 0, "seconds": 42.7, "max_memory_gb": 17.12, "ce": 0.022366295801475644, "kl": 0.6605428682960337, "context": 0.35263771697646007}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 2.8028736114501953, "loss_scale": 4.0, "retries": 0, "seconds": 44.0, "max_memory_gb": 17.12, "ce": 0.6039610191946849, "kl": 0.4515535233076662, "context": 0.8297377629205585}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 0.8608158826828003, "loss_scale": 4.0, "retries": 0, "seconds": 45.4, "max_memory_gb": 17.12, "ce": 0.037482623010873795, "kl": 0.02266517975658644, "context": 0.04881521168863401}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.026722464710474014, "loss_scale": 4.0, "retries": 0, "seconds": 46.7, "max_memory_gb": 17.12, "ce": 0.0010935252321360167, "kl": 0.0009737284162838478, "context": 0.0015803894129930995}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.032690536230802536, "loss_scale": 4.0, "retries": 0, "seconds": 48.0, "max_memory_gb": 17.13, "ce": 0.0013194378261687234, "kl": 0.00034565124406071845, "context": 0.0014922634582035244}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.863223671913147, "loss_scale": 4.0, "retries": 0, "seconds": 49.3, "max_memory_gb": 17.13, "ce": 0.04556744311412331, "kl": 0.08211163318992476, "context": 0.08662325983459596}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 1.1582874059677124, "loss_scale": 4.0, "retries": 0, "seconds": 50.7, "max_memory_gb": 17.13, "ce": 0.1456453690516355, "kl": 0.027706041033752626, "context": 0.15949839230597718}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 0.8820712566375732, "loss_scale": 4.0, "retries": 0, "seconds": 52.0, "max_memory_gb": 17.13, "ce": 0.04198489863483701, "kl": 0.3954161558067426, "context": 0.23969297637813725}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.6677936315536499, "loss_scale": 4.0, "retries": 0, "seconds": 53.4, "max_memory_gb": 17.13, "ce": 0.020298893039580435, "kl": 0.07773834053659812, "context": 0.05916806450113654}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 2.1450626850128174, "loss_scale": 4.0, "retries": 0, "seconds": 54.6, "max_memory_gb": 17.13, "ce": 0.2530091896187514, "kl": 0.20315331080928445, "context": 0.3545858508441597}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.4348502457141876, "loss_scale": 4.0, "retries": 0, "seconds": 56.0, "max_memory_gb": 17.13, "ce": 0.021131862042238936, "kl": 0.005773921293439344, "context": 0.024018823161895853}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 0.43995237350463867, "loss_scale": 4.0, "retries": 0, "seconds": 57.3, "max_memory_gb": 17.13, "ce": 0.018629722122568637, "kl": 0.04195809690281749, "context": 0.03960877051576972}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 1.1627652645111084, "loss_scale": 4.0, "retries": 0, "seconds": 58.7, "max_memory_gb": 17.13, "ce": 0.014016394707141444, "kl": 0.3661857577098999, "context": 0.19710927590494975}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 5.395786285400391, "loss_scale": 4.0, "retries": 0, "seconds": 60.0, "max_memory_gb": 17.13, "ce": 0.22531071270350367, "kl": 0.27640416884969454, "context": 0.36351279728114605}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 3.5683491230010986, "loss_scale": 4.0, "retries": 0, "seconds": 61.4, "max_memory_gb": 17.13, "ce": 0.34975308243883774, "kl": 0.034045205160509795, "context": 0.3667756929062307}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 4.100336074829102, "loss_scale": 4.0, "retries": 0, "seconds": 62.6, "max_memory_gb": 17.13, "ce": 0.27423756005009636, "kl": 0.04799546232970897, "context": 0.29823529213899747}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 0.898023247718811, "loss_scale": 4.0, "retries": 0, "seconds": 64.0, "max_memory_gb": 17.13, "ce": 0.20464196358807385, "kl": 0.03715942435519537, "context": 0.2232216777629219}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.44523435831069946, "loss_scale": 4.0, "retries": 0, "seconds": 65.3, "max_memory_gb": 17.13, "ce": 0.02693548204842955, "kl": 0.010005445772549137, "context": 0.031938205007463694}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.6183909177780151, "loss_scale": 4.0, "retries": 0, "seconds": 66.7, "max_memory_gb": 17.13, "ce": 0.09601731254952028, "kl": 0.0226526153874147, "context": 0.10734362108632922}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 1.8100526332855225, "loss_scale": 4.0, "retries": 0, "seconds": 67.9, "max_memory_gb": 17.13, "ce": 0.1209487268642988, "kl": 0.09678127616371057, "context": 0.16933936477289535}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.5771167874336243, "loss_scale": 4.0, "retries": 0, "seconds": 69.3, "max_memory_gb": 17.13, "ce": 0.026620906312018633, "kl": 0.07822663936531171, "context": 0.06573422392830253}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 2.5895426273345947, "loss_scale": 4.0, "retries": 0, "seconds": 70.6, "max_memory_gb": 17.13, "ce": 0.11131964530795813, "kl": 0.21832333644852042, "context": 0.22048131364863366}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.3685033619403839, "loss_scale": 4.0, "retries": 0, "seconds": 72.0, "max_memory_gb": 17.13, "ce": 0.01386795087455539, "kl": 0.012988540932838077, "context": 0.02036222116657882}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 0.3368702232837677, "loss_scale": 4.0, "retries": 0, "seconds": 73.3, "max_memory_gb": 17.13, "ce": 0.012652508565224707, "kl": 0.022488529881229624, "context": 0.023896773578599095}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 1.8621623516082764, "loss_scale": 4.0, "retries": 0, "seconds": 74.5, "max_memory_gb": 17.13, "ce": 0.0686067714705132, "kl": 0.06057754851019581, "context": 0.09889554338587914}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 9.30930233001709, "loss_scale": 4.0, "retries": 0, "seconds": 75.9, "max_memory_gb": 17.13, "ce": 0.7590310916057206, "kl": 0.5135463995561622, "context": 1.0158043214178178}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 1.4933135509490967, "loss_scale": 4.0, "retries": 0, "seconds": 77.2, "max_memory_gb": 17.13, "ce": 0.025197813287377357, "kl": 0.07650927110535122, "context": 0.06345245071133832}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.6327658295631409, "loss_scale": 4.0, "retries": 0, "seconds": 78.6, "max_memory_gb": 17.13, "ce": 0.056641196948476136, "kl": 0.013779302265902516, "context": 0.06353084847796708}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 1.4512836933135986, "loss_scale": 4.0, "retries": 0, "seconds": 79.9, "max_memory_gb": 17.13, "ce": 0.03071683639427647, "kl": 0.4350909474887885, "context": 0.24826231290353462}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.21893788874149323, "loss_scale": 4.0, "retries": 0, "seconds": 81.3, "max_memory_gb": 17.13, "ce": 0.010747363914560992, "kl": 0.0011263727627692788, "context": 0.011310550624330062}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 2.6459603309631348, "loss_scale": 4.0, "retries": 0, "seconds": 82.5, "max_memory_gb": 17.13, "ce": 0.061812704283511266, "kl": 0.07631916381069459, "context": 0.09997228666907176}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 0.4472896158695221, "loss_scale": 4.0, "retries": 0, "seconds": 84.0, "max_memory_gb": 17.13, "ce": 0.0057759789924602956, "kl": 0.07156018107343698, "context": 0.04155607134453021}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.5655093193054199, "loss_scale": 4.0, "retries": 0, "seconds": 85.2, "max_memory_gb": 17.13, "ce": 0.006157651456305757, "kl": 0.11853259145573247, "context": 0.06542394674033858}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 1.7582764625549316, "loss_scale": 4.0, "retries": 0, "seconds": 86.7, "max_memory_gb": 17.13, "ce": 0.12957554702006746, "kl": 0.12363599930949931, "context": 0.19139354620710947}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.7488131523132324, "loss_scale": 4.0, "retries": 0, "seconds": 87.9, "max_memory_gb": 17.13, "ce": 0.041174481440975796, "kl": 0.06709374220736208, "context": 0.07472135184070794}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 4.499373435974121, "loss_scale": 4.0, "retries": 0, "seconds": 89.3, "max_memory_gb": 17.13, "ce": 0.37679631664650515, "kl": 0.014239461277611554, "context": 0.3839160355273634}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 9.146169662475586, "loss_scale": 4.0, "retries": 0, "seconds": 90.6, "max_memory_gb": 17.13, "ce": 0.11920454014762072, "kl": 0.13387422726918885, "context": 0.18614166123734321}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.21567726135253906, "loss_scale": 4.0, "retries": 0, "seconds": 92.0, "max_memory_gb": 17.13, "ce": 0.013118291510181734, "kl": 0.001998773021114175, "context": 0.01411767779791262}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.40855079889297485, "loss_scale": 4.0, "retries": 0, "seconds": 93.3, "max_memory_gb": 17.13, "ce": 0.030541222164174542, "kl": 0.016007990339858225, "context": 0.03854521682660561}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 1.409929633140564, "loss_scale": 4.0, "retries": 0, "seconds": 94.7, "max_memory_gb": 17.13, "ce": 0.10280103520199191, "kl": 0.05378511699382216, "context": 0.12969359388807788}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 3.72822904586792, "loss_scale": 4.0, "retries": 0, "seconds": 96.0, "max_memory_gb": 17.13, "ce": 0.37740840039623436, "kl": 0.13638833875484124, "context": 0.44560256943805143}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 3.9871439933776855, "loss_scale": 4.0, "retries": 0, "seconds": 97.4, "max_memory_gb": 17.13, "ce": 0.6357244109385647, "kl": 0.014813991554547101, "context": 0.6431314329383895}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.05875064432621002, "loss_scale": 4.0, "retries": 0, "seconds": 98.7, "max_memory_gb": 17.13, "ce": 0.002384380451985635, "kl": 0.0032679459400242195, "context": 0.004018353356514126}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 3.1182022094726562, "loss_scale": 4.0, "retries": 0, "seconds": 100.1, "max_memory_gb": 17.13, "ce": 0.3337667909290758, "kl": 0.01426925972191384, "context": 0.3409014210192254}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.4474364519119263, "loss_scale": 4.0, "retries": 0, "seconds": 101.4, "max_memory_gb": 17.13, "ce": 0.006229056423762813, "kl": 0.10787128018273506, "context": 0.06016469554742798}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.5709292888641357, "loss_scale": 4.0, "retries": 0, "seconds": 102.8, "max_memory_gb": 17.13, "ce": 0.041913890301657375, "kl": 0.1740446149822219, "context": 0.12893620209797518}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 1.3291584253311157, "loss_scale": 4.0, "retries": 0, "seconds": 104.1, "max_memory_gb": 17.13, "ce": 0.005475782949361019, "kl": 0.23435902927303687, "context": 0.12265530065633357}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.9177254438400269, "loss_scale": 4.0, "retries": 0, "seconds": 105.5, "max_memory_gb": 17.13, "ce": 0.004387556866277009, "kl": 0.12732075556550626, "context": 0.06804793540504761}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.8301851153373718, "loss_scale": 4.0, "retries": 0, "seconds": 106.8, "max_memory_gb": 17.13, "ce": 0.00842420177650638, "kl": 0.1499984936817782, "context": 0.08342344666016288}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.8706344366073608, "loss_scale": 4.0, "retries": 0, "seconds": 108.1, "max_memory_gb": 17.13, "ce": 0.015058405275340192, "kl": 0.3365595322156878, "context": 0.18333816684025805}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 1.3159775733947754, "loss_scale": 4.0, "retries": 0, "seconds": 109.4, "max_memory_gb": 17.13, "ce": 0.21124358419547207, "kl": 0.043992847547542624, "context": 0.23324000051434268}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 3.426853895187378, "loss_scale": 4.0, "retries": 0, "seconds": 110.8, "max_memory_gb": 17.13, "ce": 0.6828800288567436, "kl": 0.04990683196228929, "context": 0.7078334616599022}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 1.2970634698867798, "loss_scale": 4.0, "retries": 0, "seconds": 112.1, "max_memory_gb": 17.13, "ce": 0.0006986176376813091, "kl": 0.18668733116192016, "context": 0.0940422802377725}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 0.8514999747276306, "loss_scale": 4.0, "retries": 0, "seconds": 113.5, "max_memory_gb": 17.13, "ce": 0.057796533685177565, "kl": 0.08500262230518274, "context": 0.10029784601647407}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 0.7131004333496094, "loss_scale": 4.0, "retries": 0, "seconds": 114.8, "max_memory_gb": 17.13, "ce": 0.02980207424843684, "kl": 0.1787470723502338, "context": 0.11917560803703964}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 1.050026297569275, "loss_scale": 4.0, "retries": 0, "seconds": 116.0, "max_memory_gb": 17.13, "ce": 0.055219320725882426, "kl": 0.04665776948377243, "context": 0.0785482054707245}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.43437835574150085, "loss_scale": 4.0, "retries": 0, "seconds": 117.4, "max_memory_gb": 17.13, "ce": 0.020369874036987312, "kl": 0.01678036848534248, "context": 0.02876005828147754}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.7499861121177673, "loss_scale": 4.0, "retries": 0, "seconds": 118.8, "max_memory_gb": 17.13, "ce": 0.0456734665640397, "kl": 0.024164958511391887, "context": 0.057755947709665634}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.4159233868122101, "loss_scale": 4.0, "retries": 0, "seconds": 120.0, "max_memory_gb": 17.13, "ce": 0.026026738560176454, "kl": 0.0005353915912564844, "context": 0.026294434908777475}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.2699027955532074, "loss_scale": 4.0, "retries": 0, "seconds": 121.4, "max_memory_gb": 17.13, "ce": 0.010107883106684312, "kl": 0.022843012397061102, "context": 0.02152938995277509}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.033226873725652695, "loss_scale": 4.0, "retries": 0, "seconds": 122.7, "max_memory_gb": 17.13, "ce": 0.0008869464945746586, "kl": 0.003026276090167812, "context": 0.002400084587861784}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 0.5333235263824463, "loss_scale": 4.0, "retries": 0, "seconds": 123.9, "max_memory_gb": 17.13, "ce": 0.018048802157863975, "kl": 0.03474648523842916, "context": 0.03542204573750496}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 1.0228638648986816, "loss_scale": 4.0, "retries": 0, "seconds": 125.3, "max_memory_gb": 17.13, "ce": 0.09308014895941596, "kl": 0.016488784225657582, "context": 0.10132454265840352}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 3.013949394226074, "loss_scale": 4.0, "retries": 0, "seconds": 126.6, "max_memory_gb": 17.13, "ce": 0.1715169987292029, "kl": 0.358490094949957, "context": 0.35076203965581954}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 2.381037950515747, "loss_scale": 4.0, "retries": 0, "seconds": 127.9, "max_memory_gb": 17.13, "ce": 0.16118554803688312, "kl": 0.41272682091221213, "context": 0.36754897714126855}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 0.7855842113494873, "loss_scale": 4.0, "retries": 0, "seconds": 129.2, "max_memory_gb": 17.13, "ce": 0.0316536626778543, "kl": 0.09116652724333107, "context": 0.07723692618310452}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 1.0153700113296509, "loss_scale": 4.0, "retries": 0, "seconds": 130.6, "max_memory_gb": 17.13, "ce": 0.007762292516417801, "kl": 0.164252882488654, "context": 0.0898887359071523}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.2786003053188324, "loss_scale": 4.0, "retries": 0, "seconds": 131.8, "max_memory_gb": 17.13, "ce": 0.0020251288369763643, "kl": 0.029743363658781163, "context": 0.01689681046991609}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.30575037002563477, "loss_scale": 4.0, "retries": 0, "seconds": 133.2, "max_memory_gb": 17.13, "ce": 0.019026207119168248, "kl": 0.02858056388959085, "context": 0.03331648876337567}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.6054255962371826, "loss_scale": 4.0, "retries": 0, "seconds": 134.5, "max_memory_gb": 17.13, "ce": 0.014343438026116928, "kl": 0.048652384537490434, "context": 0.03866963060863782}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.14183121919631958, "loss_scale": 4.0, "retries": 0, "seconds": 135.9, "max_memory_gb": 17.13, "ce": 0.00817694334546104, "kl": 0.0033807407999120187, "context": 0.009867314147413708}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 1.2369767427444458, "loss_scale": 4.0, "retries": 0, "seconds": 137.1, "max_memory_gb": 17.13, "ce": 0.15309606034679746, "kl": 0.046061311234552704, "context": 0.17612671234564914}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.23259319365024567, "loss_scale": 4.0, "retries": 0, "seconds": 138.5, "max_memory_gb": 17.13, "ce": 0.004368505498860031, "kl": 0.022797406147788024, "context": 0.01576720833327272}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 1.7320783138275146, "loss_scale": 4.0, "retries": 0, "seconds": 139.8, "max_memory_gb": 17.13, "ce": 0.22186079318635166, "kl": 0.06991344615380513, "context": 0.25681752688251436}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 1.1448931694030762, "loss_scale": 4.0, "retries": 0, "seconds": 141.1, "max_memory_gb": 17.13, "ce": 0.1416455069265794, "kl": 0.0041939088914659806, "context": 0.14374245976796374}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 11.09537410736084, "loss_scale": 4.0, "retries": 0, "seconds": 142.4, "max_memory_gb": 17.13, "ce": 0.15031088800969883, "kl": 0.4731933192539657, "context": 0.3869075534566946}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 3.5576164722442627, "loss_scale": 4.0, "retries": 0, "seconds": 143.8, "max_memory_gb": 17.13, "ce": 0.20078629664203618, "kl": 0.015323220526624937, "context": 0.20844790498085786}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 1.947852373123169, "loss_scale": 4.0, "retries": 0, "seconds": 145.0, "max_memory_gb": 17.13, "ce": 0.20521797698165756, "kl": 0.09534781343973009, "context": 0.25289189092291053}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 2.4882760047912598, "loss_scale": 4.0, "retries": 0, "seconds": 146.4, "max_memory_gb": 17.13, "ce": 0.27968149607477244, "kl": 0.002689896817173576, "context": 0.28102643828606233}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 1.3646959066390991, "loss_scale": 4.0, "retries": 0, "seconds": 147.7, "max_memory_gb": 17.13, "ce": 0.0408956611645408, "kl": 0.40231223324497023, "context": 0.24205177772091702}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 1.7711434364318848, "loss_scale": 4.0, "retries": 0, "seconds": 149.1, "max_memory_gb": 17.13, "ce": 0.06368156498501776, "kl": 0.0028812665996156284, "context": 0.0651221977896057}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.018725715577602386, "loss_scale": 4.0, "retries": 0, "seconds": 150.3, "max_memory_gb": 17.13, "ce": 0.0005458719424495939, "kl": 0.0012110661004385292, "context": 0.0011514050092955586}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 3.6337289810180664, "loss_scale": 4.0, "retries": 0, "seconds": 151.7, "max_memory_gb": 17.13, "ce": 0.11030869647220243, "kl": 0.21844993757258635, "context": 0.21953367270180024}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 1.7223554849624634, "loss_scale": 4.0, "retries": 0, "seconds": 153.0, "max_memory_gb": 17.13, "ce": 0.552722613694641, "kl": 0.10455519708557404, "context": 0.6050002196570858}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.3533589541912079, "loss_scale": 4.0, "retries": 0, "seconds": 154.3, "max_memory_gb": 17.13, "ce": 0.018846496590413153, "kl": 0.0029691109630221035, "context": 0.020331052161054686}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 6.507090091705322, "loss_scale": 4.0, "retries": 0, "seconds": 155.7, "max_memory_gb": 17.13, "ce": 0.289413882419467, "kl": 0.023747052939143032, "context": 0.3012874126434326}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 2.18755841255188, "loss_scale": 4.0, "retries": 0, "seconds": 157.0, "max_memory_gb": 17.13, "ce": 0.11185617464070674, "kl": 0.000841567602037685, "context": 0.1122769578214502}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 3.102414608001709, "loss_scale": 4.0, "retries": 0, "seconds": 158.3, "max_memory_gb": 17.13, "ce": 0.49064968915809004, "kl": 0.026745959965410293, "context": 0.5040226523560705}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 3.2162821292877197, "loss_scale": 4.0, "retries": 0, "seconds": 159.6, "max_memory_gb": 17.13, "ce": 0.16563190740271239, "kl": 1.0110962457329151, "context": 0.671180025616195}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 1.488442301750183, "loss_scale": 4.0, "retries": 0, "seconds": 161.0, "max_memory_gb": 17.13, "ce": 0.06604898981822771, "kl": 0.0050904475924653525, "context": 0.06859420986438636}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 1.5743783712387085, "loss_scale": 4.0, "retries": 0, "seconds": 162.2, "max_memory_gb": 17.13, "ce": 0.015501288977247896, "kl": 0.08717949120446633, "context": 0.059091036455356516}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.8055706024169922, "loss_scale": 4.0, "retries": 0, "seconds": 163.6, "max_memory_gb": 17.13, "ce": 0.028143875792011386, "kl": 0.06429365469216464, "context": 0.060290705176157644}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 0.7327496409416199, "loss_scale": 4.0, "retries": 0, "seconds": 164.9, "max_memory_gb": 17.13, "ce": 0.005108616431243718, "kl": 0.09497581877803896, "context": 0.05259652464883402}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.2109478861093521, "loss_scale": 4.0, "retries": 0, "seconds": 166.5, "max_memory_gb": 17.13, "ce": 0.009028853572090156, "kl": 0.005198389762881561, "context": 0.011628048931015655}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 1.3743466138839722, "loss_scale": 4.0, "retries": 0, "seconds": 167.9, "max_memory_gb": 17.13, "ce": 0.0290415181661956, "kl": 0.602673410866676, "context": 0.33037823288032087}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 5.392301559448242, "loss_scale": 4.0, "retries": 0, "seconds": 169.4, "max_memory_gb": 17.13, "ce": 0.06552570729400031, "kl": 0.21186235256755026, "context": 0.17145687801530585}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 2.7653493881225586, "loss_scale": 4.0, "retries": 0, "seconds": 170.9, "max_memory_gb": 17.13, "ce": 0.5836455677927006, "kl": 0.19979450218052808, "context": 0.6835427890764549}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.3648603856563568, "loss_scale": 4.0, "retries": 0, "seconds": 172.4, "max_memory_gb": 17.13, "ce": 0.0014222212048480287, "kl": 0.04338743015250657, "context": 0.02311593567719683}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.6805883049964905, "loss_scale": 4.0, "retries": 0, "seconds": 173.8, "max_memory_gb": 17.13, "ce": 0.038646664572297595, "kl": 0.04541881244222168, "context": 0.06135606946190819}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 7.76890230178833, "loss_scale": 4.0, "retries": 0, "seconds": 175.4, "max_memory_gb": 17.13, "ce": 0.2614739799100789, "kl": 0.16701567792870264, "context": 0.34498180398441036}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 1.8721225261688232, "loss_scale": 4.0, "retries": 0, "seconds": 176.8, "max_memory_gb": 17.13, "ce": 0.04876379485358484, "kl": 0.084165450243745, "context": 0.0908465160173364}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 0.8270601034164429, "loss_scale": 4.0, "retries": 0, "seconds": 178.3, "max_memory_gb": 17.13, "ce": 0.15239616570761427, "kl": 0.04494078439893201, "context": 0.1748665606137365}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.025453567504882812, "loss_scale": 4.0, "retries": 0, "seconds": 179.7, "max_memory_gb": 17.13, "ce": 0.0019178571383235976, "kl": 0.00014061648380447878, "context": 0.0019881653861375526}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.10453510284423828, "loss_scale": 4.0, "retries": 0, "seconds": 181.3, "max_memory_gb": 17.13, "ce": 0.00946218182798475, "kl": 0.001098974418709986, "context": 0.010011669422965497}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 2.040010452270508, "loss_scale": 4.0, "retries": 0, "seconds": 182.7, "max_memory_gb": 17.13, "ce": 0.015731548439362086, "kl": 0.09118411893700795, "context": 0.06132360885385424}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 1.579929232597351, "loss_scale": 4.0, "retries": 0, "seconds": 184.2, "max_memory_gb": 17.13, "ce": 0.3119420766015537, "kl": 0.43015362552250735, "context": 0.5270188729045913}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 0.2529035210609436, "loss_scale": 4.0, "retries": 0, "seconds": 185.6, "max_memory_gb": 17.13, "ce": 0.009879401339276228, "kl": 0.0041970707243308425, "context": 0.011977936926996335}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 5.01873254776001, "loss_scale": 4.0, "retries": 0, "seconds": 187.2, "max_memory_gb": 17.13, "ce": 0.43958834509248845, "kl": 0.3317643511982169, "context": 0.6054705019341782}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.22851715981960297, "loss_scale": 4.0, "retries": 0, "seconds": 188.6, "max_memory_gb": 17.13, "ce": 0.01687642182514537, "kl": 0.0020166671311017126, "context": 0.017884755099657923}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 2.13039231300354, "loss_scale": 4.0, "retries": 0, "seconds": 190.2, "max_memory_gb": 17.13, "ce": 0.3208982391806785, "kl": 0.012478058162287198, "context": 0.327137272717664}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.30648544430732727, "loss_scale": 4.0, "retries": 0, "seconds": 191.6, "max_memory_gb": 17.13, "ce": 0.015205512921966147, "kl": 0.0059180155767535325, "context": 0.018164521403377876}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 2.1064248085021973, "loss_scale": 4.0, "retries": 0, "seconds": 193.1, "max_memory_gb": 17.13, "ce": 0.13967722316738218, "kl": 0.22315571550279856, "context": 0.25125507451593876}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 0.5080749988555908, "loss_scale": 4.0, "retries": 0, "seconds": 194.6, "max_memory_gb": 17.13, "ce": 0.0032868957932805642, "kl": 0.06836539773939876, "context": 0.037469595758011565}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 1.040277123451233, "loss_scale": 4.0, "retries": 0, "seconds": 196.1, "max_memory_gb": 17.13, "ce": 0.1576264367904514, "kl": 0.027246003272011876, "context": 0.17124943889211863}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.7318321466445923, "loss_scale": 4.0, "retries": 0, "seconds": 197.5, "max_memory_gb": 17.13, "ce": 0.003861231725750258, "kl": 0.06860735523150652, "context": 0.03816490899771452}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.19578224420547485, "loss_scale": 4.0, "retries": 0, "seconds": 198.9, "max_memory_gb": 17.13, "ce": 0.01738987425051164, "kl": 0.004159560814969154, "context": 0.01946965452225413}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.0428585484623909, "loss_scale": 4.0, "retries": 0, "seconds": 200.5, "max_memory_gb": 17.13, "ce": 0.0031972884899005294, "kl": 0.0010221412521786988, "context": 0.0037083590286783874}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.470228672027588, "loss_scale": 4.0, "retries": 0, "seconds": 202.0, "max_memory_gb": 17.13, "ce": 0.07205848525336478, "kl": 0.005384549709560815, "context": 0.07475076231639832}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 2.154482126235962, "loss_scale": 4.0, "retries": 0, "seconds": 203.3, "max_memory_gb": 17.13, "ce": 0.295776417828165, "kl": 0.11289268867403734, "context": 0.35222275147680193}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 1.634013295173645, "loss_scale": 4.0, "retries": 0, "seconds": 204.7, "max_memory_gb": 17.13, "ce": 0.090703103775013, "kl": 0.24367913328660507, "context": 0.2125426711718319}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 1.500379204750061, "loss_scale": 4.0, "retries": 0, "seconds": 205.9, "max_memory_gb": 17.13, "ce": 0.002866114878997905, "kl": 0.23625727814351194, "context": 0.12099475115610403}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.040607377886772156, "loss_scale": 4.0, "retries": 0, "seconds": 207.2, "max_memory_gb": 17.13, "ce": 0.0032088544394355267, "kl": 0.0005618006125587272, "context": 0.003489754773909226}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 1.3204718828201294, "loss_scale": 4.0, "retries": 0, "seconds": 208.6, "max_memory_gb": 17.13, "ce": 0.07659600328770466, "kl": 0.003608204617194133, "context": 0.07840010733343661}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.10712079703807831, "loss_scale": 4.0, "retries": 0, "seconds": 209.8, "max_memory_gb": 17.13, "ce": 0.0032693945977371186, "kl": 0.007817969904863276, "context": 0.007178379571996629}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.10814491659402847, "loss_scale": 4.0, "retries": 0, "seconds": 211.2, "max_memory_gb": 17.13, "ce": 0.009151604765065713, "kl": 0.001564156318863752, "context": 0.009933683097187895}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.43305912613868713, "loss_scale": 4.0, "retries": 0, "seconds": 212.5, "max_memory_gb": 17.13, "ce": 0.09467491290706675, "kl": 0.05913956107724516, "context": 0.12424468973767944}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 1.185206413269043, "loss_scale": 4.0, "retries": 0, "seconds": 213.9, "max_memory_gb": 17.13, "ce": 0.10694865180994384, "kl": 0.01701515448075952, "context": 0.11545622962876223}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.039667628705501556, "loss_scale": 4.0, "retries": 0, "seconds": 215.1, "max_memory_gb": 17.13, "ce": 0.0020132897043367848, "kl": 0.0016356880696548615, "context": 0.002831133737345226}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.3039565086364746, "loss_scale": 4.0, "retries": 0, "seconds": 216.5, "max_memory_gb": 17.13, "ce": 0.00556348217651248, "kl": 0.30744962947210297, "context": 0.15928829018957913}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.3340764045715332, "loss_scale": 4.0, "retries": 0, "seconds": 217.8, "max_memory_gb": 17.13, "ce": 0.010145152686163783, "kl": 0.025508468866974, "context": 0.022899387346114963}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.35002195835113525, "loss_scale": 4.0, "retries": 0, "seconds": 219.1, "max_memory_gb": 17.13, "ce": 0.03304003580160497, "kl": 0.007063325863327918, "context": 0.03657169931102544}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 1.0296566486358643, "loss_scale": 4.0, "retries": 0, "seconds": 220.4, "max_memory_gb": 17.13, "ce": 0.0730999780644197, "kl": 0.0582308663870208, "context": 0.10221541381906718}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 2.147688150405884, "loss_scale": 4.0, "retries": 0, "seconds": 221.8, "max_memory_gb": 17.13, "ce": 0.009339501599242794, "kl": 0.09128719607906532, "context": 0.054983097783406265}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 0.5856384038925171, "loss_scale": 4.0, "retries": 0, "seconds": 223.1, "max_memory_gb": 17.13, "ce": 0.08183188701514155, "kl": 0.07092017815830332, "context": 0.11729197599925101}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.8641158938407898, "loss_scale": 4.0, "retries": 0, "seconds": 224.4, "max_memory_gb": 17.13, "ce": 0.12119445390999317, "kl": 0.017907907837070525, "context": 0.13014841149561107}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.2231028527021408, "loss_scale": 4.0, "retries": 0, "seconds": 225.7, "max_memory_gb": 17.13, "ce": 0.0024512262316420674, "kl": 0.030868669613937527, "context": 0.01788556156679988}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.17406676709651947, "loss_scale": 4.0, "retries": 0, "seconds": 227.1, "max_memory_gb": 17.13, "ce": 0.010487016930710524, "kl": 0.009856957571173552, "context": 0.015415496192872524}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.14372298121452332, "loss_scale": 4.0, "retries": 0, "seconds": 228.4, "max_memory_gb": 17.13, "ce": 0.005071168023278005, "kl": 0.004392038679725374, "context": 0.007267187378602102}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.5004560351371765, "loss_scale": 4.0, "retries": 0, "seconds": 229.7, "max_memory_gb": 17.13, "ce": 0.01730834465706721, "kl": 0.06843235889391508, "context": 0.05152452288893983}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 1.6907339096069336, "loss_scale": 4.0, "retries": 0, "seconds": 231.0, "max_memory_gb": 17.13, "ce": 0.014029432204551995, "kl": 0.1445583265485766, "context": 0.0863085954770213}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 1.1379070281982422, "loss_scale": 4.0, "retries": 0, "seconds": 232.4, "max_memory_gb": 17.13, "ce": 0.03865140004927525, "kl": 0.2615208665592945, "context": 0.16941183703602292}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 1.7461364269256592, "loss_scale": 4.0, "retries": 0, "seconds": 233.6, "max_memory_gb": 17.13, "ce": 0.0932680258556502, "kl": 0.018321241055673454, "context": 0.10242864448809996}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 1.8717331886291504, "loss_scale": 4.0, "retries": 0, "seconds": 235.0, "max_memory_gb": 17.13, "ce": 0.1900381119339727, "kl": 0.02034165631630458, "context": 0.20020893646869808}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.37052658200263977, "loss_scale": 4.0, "retries": 0, "seconds": 236.3, "max_memory_gb": 17.13, "ce": 0.018784912783303298, "kl": 0.016667678763042204, "context": 0.027118752113892697}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 2.9795897006988525, "loss_scale": 4.0, "retries": 0, "seconds": 237.5, "max_memory_gb": 17.13, "ce": 0.012551306695968378, "kl": 0.1900839429035841, "context": 0.10759327714913525}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 0.2867106795310974, "loss_scale": 4.0, "retries": 0, "seconds": 238.9, "max_memory_gb": 17.13, "ce": 0.01968016181490384, "kl": 0.00487739087054706, "context": 0.0221188580035232}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.7405928373336792, "loss_scale": 4.0, "retries": 0, "seconds": 240.2, "max_memory_gb": 17.13, "ce": 0.015513708582147956, "kl": 0.24015168608457316, "context": 0.13558955467306077}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 4.321486473083496, "loss_scale": 4.0, "retries": 0, "seconds": 241.5, "max_memory_gb": 17.13, "ce": 0.19457522645097924, "kl": 0.12500778965477366, "context": 0.2570791273319628}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.8456240892410278, "loss_scale": 4.0, "retries": 0, "seconds": 242.8, "max_memory_gb": 17.13, "ce": 0.0011776389947044663, "kl": 0.08111006033868762, "context": 0.04173266838188283}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 0.7220883369445801, "loss_scale": 4.0, "retries": 0, "seconds": 244.2, "max_memory_gb": 17.13, "ce": 0.05911607257439755, "kl": 0.09232027659163577, "context": 0.10527621087385342}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.8277886509895325, "loss_scale": 4.0, "retries": 0, "seconds": 245.6, "max_memory_gb": 17.13, "ce": 0.20648672290190007, "kl": 0.016266343343886547, "context": 0.2146198995760642}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 2.8745007514953613, "loss_scale": 4.0, "retries": 0, "seconds": 247.1, "max_memory_gb": 17.13, "ce": 0.37847612623590976, "kl": 0.12454395223176107, "context": 0.44074811215978116}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.7591635584831238, "loss_scale": 4.0, "retries": 0, "seconds": 248.5, "max_memory_gb": 17.13, "ce": 0.08082174346782267, "kl": 0.09778108043155953, "context": 0.12971228065725882}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.2688825726509094, "loss_scale": 4.0, "retries": 0, "seconds": 250.1, "max_memory_gb": 17.13, "ce": 0.008372591590159573, "kl": 0.0011653318942990154, "context": 0.00895525769010419}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.024410678073763847, "loss_scale": 4.0, "retries": 0, "seconds": 251.5, "max_memory_gb": 17.13, "ce": 0.0014260194438975304, "kl": 0.0007820226237527095, "context": 0.001817030759411864}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.019597122445702553, "loss_scale": 4.0, "retries": 0, "seconds": 253.0, "max_memory_gb": 17.13, "ce": 0.0011312643036944792, "kl": 0.0006923629462107783, "context": 0.0014774457886232994}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 0.09985262155532837, "loss_scale": 4.0, "retries": 0, "seconds": 254.4, "max_memory_gb": 17.13, "ce": 0.0029317203516256995, "kl": 0.009111429622862488, "context": 0.007487435243092477}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 1.9093104600906372, "loss_scale": 4.0, "retries": 0, "seconds": 256.0, "max_memory_gb": 17.13, "ce": 0.22170904527592938, "kl": 0.008816337267489871, "context": 0.22611720878921915}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.3527711629867554, "loss_scale": 4.0, "retries": 0, "seconds": 257.4, "max_memory_gb": 17.13, "ce": 0.048044004128314555, "kl": 0.11307642083193059, "context": 0.1045822182786651}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 0.8290587663650513, "loss_scale": 4.0, "retries": 0, "seconds": 259.0, "max_memory_gb": 17.13, "ce": 0.07608569205876847, "kl": 0.015597130871526588, "context": 0.08388425778503006}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 1.1658164262771606, "loss_scale": 4.0, "retries": 0, "seconds": 260.4, "max_memory_gb": 17.13, "ce": 0.04886127746431157, "kl": 0.022963316610912443, "context": 0.060342934615619015}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.05237820744514465, "loss_scale": 4.0, "retries": 0, "seconds": 261.9, "max_memory_gb": 17.13, "ce": 0.0028851320894318633, "kl": 0.00230928766541183, "context": 0.004039775871206075}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.964449167251587, "loss_scale": 4.0, "retries": 0, "seconds": 263.3, "max_memory_gb": 17.13, "ce": 0.32313553025596775, "kl": 0.4009593000228051, "context": 0.5236151750432327}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.5247610211372375, "loss_scale": 4.0, "retries": 0, "seconds": 264.9, "max_memory_gb": 17.13, "ce": 0.015882731908277492, "kl": 0.058753178995175404, "context": 0.04525932233809726}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.6224572658538818, "loss_scale": 4.0, "retries": 0, "seconds": 266.3, "max_memory_gb": 17.13, "ce": 0.006837175504188053, "kl": 0.07978499172349984, "context": 0.04672967077931389}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.027928199619054794, "loss_scale": 4.0, "retries": 0, "seconds": 267.9, "max_memory_gb": 17.13, "ce": 0.001574426394654438, "kl": 0.0004913442535325885, "context": 0.0018200984923169017}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.04730117321014404, "loss_scale": 4.0, "retries": 0, "seconds": 269.3, "max_memory_gb": 17.13, "ce": 0.0019978445634478703, "kl": 0.0007185820593633707, "context": 0.0023571355995954946}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.1322345733642578, "loss_scale": 4.0, "retries": 0, "seconds": 270.8, "max_memory_gb": 17.13, "ce": 0.0005719465152651537, "kl": 0.007891616027336568, "context": 0.004517754630796844}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 1.0068010091781616, "loss_scale": 4.0, "retries": 0, "seconds": 272.2, "max_memory_gb": 17.13, "ce": 0.0013818450624967227, "kl": 0.29259453182157813, "context": 0.1476791145541938}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 3.1061973571777344, "loss_scale": 4.0, "retries": 0, "seconds": 273.8, "max_memory_gb": 17.13, "ce": 0.10529190684610512, "kl": 0.3135192018962698, "context": 0.26205150395981036}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.07579252868890762, "loss_scale": 4.0, "retries": 0, "seconds": 275.2, "max_memory_gb": 17.13, "ce": 0.008494207253534114, "kl": 0.0004941540209983941, "context": 0.008741284123971127}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 3.3260018825531006, "loss_scale": 4.0, "retries": 0, "seconds": 276.7, "max_memory_gb": 17.13, "ce": 0.027000220608897507, "kl": 0.10445529254502617, "context": 0.07922786613926291}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 2.2400848865509033, "loss_scale": 4.0, "retries": 0, "seconds": 278.1, "max_memory_gb": 17.13, "ce": 0.013555537821957842, "kl": 1.0111056757159531, "context": 0.5191083659883589}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.09565497189760208, "loss_scale": 4.0, "retries": 0, "seconds": 279.7, "max_memory_gb": 17.13, "ce": 0.005867802741704509, "kl": 0.0024712604990781983, "context": 0.007103433053998742}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.05526793375611305, "loss_scale": 4.0, "retries": 0, "seconds": 281.1, "max_memory_gb": 17.13, "ce": 0.003872232358844485, "kl": 0.0011901915386260953, "context": 0.004467328129976522}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.07732634991407394, "loss_scale": 4.0, "retries": 0, "seconds": 282.5, "max_memory_gb": 17.13, "ce": 0.005383635805628728, "kl": 0.003980184861575253, "context": 0.007373728469246998}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.5339237451553345, "loss_scale": 4.0, "retries": 0, "seconds": 284.1, "max_memory_gb": 17.13, "ce": 0.01692897645989433, "kl": 0.037266471445036586, "context": 0.03556221234612167}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.20838503539562225, "loss_scale": 4.0, "retries": 0, "seconds": 285.6, "max_memory_gb": 17.13, "ce": 0.005215034761931747, "kl": 0.015070773100774204, "context": 0.012750421185046434}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 1.6537716388702393, "loss_scale": 4.0, "retries": 0, "seconds": 287.0, "max_memory_gb": 17.13, "ce": 0.06325388445111457, "kl": 0.054486125518451445, "context": 0.09049694723216817}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.3031783401966095, "loss_scale": 4.0, "retries": 0, "seconds": 288.6, "max_memory_gb": 17.13, "ce": 0.013882432016544044, "kl": 0.007383962813037215, "context": 0.01757441327208653}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 0.8871079087257385, "loss_scale": 4.0, "retries": 0, "seconds": 290.0, "max_memory_gb": 17.13, "ce": 0.05490256941084226, "kl": 0.025271510263337404, "context": 0.06753832258254988}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 1.542771577835083, "loss_scale": 4.0, "retries": 0, "seconds": 291.4, "max_memory_gb": 17.13, "ce": 0.03421226445789216, "kl": 0.06010787535979034, "context": 0.06426620213824208}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 1.120873212814331, "loss_scale": 4.0, "retries": 0, "seconds": 292.9, "max_memory_gb": 17.13, "ce": 0.12794458591088187, "kl": 0.22011133772821267, "context": 0.23800025302625727}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 1.6238210201263428, "loss_scale": 4.0, "retries": 0, "seconds": 294.4, "max_memory_gb": 17.13, "ce": 0.3655391948996112, "kl": 0.5982335807057098, "context": 0.664655986300204}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.2731308043003082, "loss_scale": 4.0, "retries": 0, "seconds": 295.9, "max_memory_gb": 17.13, "ce": 0.015741336479550228, "kl": 0.004144044291024329, "context": 0.017813357801060192}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.37887337803840637, "loss_scale": 4.0, "retries": 0, "seconds": 297.3, "max_memory_gb": 17.13, "ce": 0.014806060877162963, "kl": 0.003950537357013673, "context": 0.016781329293735325}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.9270589351654053, "loss_scale": 4.0, "retries": 0, "seconds": 298.9, "max_memory_gb": 17.13, "ce": 0.0028186336276121438, "kl": 0.14988687305685744, "context": 0.07776206822018139}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 1.4445018768310547, "loss_scale": 4.0, "retries": 0, "seconds": 300.3, "max_memory_gb": 17.13, "ce": 0.0013390462027018657, "kl": 0.3798356513489125, "context": 0.191256868209166}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 5.585877895355225, "loss_scale": 4.0, "retries": 0, "seconds": 301.8, "max_memory_gb": 17.13, "ce": 0.1274512378113286, "kl": 1.1059603794419672, "context": 0.6804314235778293}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.3430938720703125, "loss_scale": 4.0, "retries": 0, "seconds": 303.2, "max_memory_gb": 17.13, "ce": 0.013217886295024073, "kl": 0.017601757614102098, "context": 0.022018765052052913}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 2.079467296600342, "loss_scale": 4.0, "retries": 0, "seconds": 304.8, "max_memory_gb": 17.13, "ce": 0.06301105039892718, "kl": 0.4454212629061658, "context": 0.28572168230311945}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 2.966099500656128, "loss_scale": 4.0, "retries": 0, "seconds": 306.2, "max_memory_gb": 17.13, "ce": 0.272556685442396, "kl": 0.02372380550514208, "context": 0.28441857505822554}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.9330137968063354, "loss_scale": 4.0, "retries": 0, "seconds": 307.7, "max_memory_gb": 17.13, "ce": 0.005427390729892068, "kl": 0.05335479194036452, "context": 0.032104785888805054}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 0.403584361076355, "loss_scale": 4.0, "retries": 0, "seconds": 309.2, "max_memory_gb": 17.13, "ce": 0.007638772658538073, "kl": 0.06135402712970972, "context": 0.03831578732933849}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.012372502125799656, "loss_scale": 4.0, "retries": 0, "seconds": 310.7, "max_memory_gb": 17.13, "ce": 0.0007657923015358392, "kl": 8.21138365125762e-05, "context": 0.0008068492170423269}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 1.7400836944580078, "loss_scale": 4.0, "retries": 0, "seconds": 312.1, "max_memory_gb": 17.13, "ce": 0.02254806491691852, "kl": 0.9102315316422391, "context": 0.4776638242183253}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.6428353786468506, "loss_scale": 4.0, "retries": 0, "seconds": 313.6, "max_memory_gb": 17.13, "ce": 0.04127220227383077, "kl": 0.18661710165906698, "context": 0.13458075281232595}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.18870633840560913, "loss_scale": 4.0, "retries": 0, "seconds": 315.0, "max_memory_gb": 17.13, "ce": 0.006960020604310557, "kl": 0.02341449367486348, "context": 0.018667267271666788}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 1.4286190271377563, "loss_scale": 4.0, "retries": 0, "seconds": 316.6, "max_memory_gb": 17.13, "ce": 0.12379958358360454, "kl": 0.10267323646803561, "context": 0.17513620178215206}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 1.4889193773269653, "loss_scale": 4.0, "retries": 0, "seconds": 318.0, "max_memory_gb": 17.13, "ce": 0.4446193376497831, "kl": 0.01334935677004978, "context": 0.45129402342718095}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.5997811555862427, "loss_scale": 4.0, "retries": 0, "seconds": 319.5, "max_memory_gb": 17.13, "ce": 0.003368406541994773, "kl": 0.07567774553263007, "context": 0.041207279511581874}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 1.5438803434371948, "loss_scale": 4.0, "retries": 0, "seconds": 320.9, "max_memory_gb": 17.13, "ce": 0.052090066456003115, "kl": 0.25661605544155464, "context": 0.1803980972908903}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.6246611475944519, "loss_scale": 4.0, "retries": 0, "seconds": 322.4, "max_memory_gb": 17.13, "ce": 0.0012675740799750201, "kl": 0.07300557972484967, "context": 0.03777036334940931}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.28480783104896545, "loss_scale": 4.0, "retries": 0, "seconds": 323.8, "max_memory_gb": 17.13, "ce": 0.03191764958319254, "kl": 0.006669426005828427, "context": 0.03525236208224669}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 3.450979709625244, "loss_scale": 4.0, "retries": 0, "seconds": 325.2, "max_memory_gb": 17.13, "ce": 0.030954059155192226, "kl": 0.27089657890610397, "context": 0.16640234272927046}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 0.6449793577194214, "loss_scale": 4.0, "retries": 0, "seconds": 326.7, "max_memory_gb": 17.13, "ce": 0.06000207775286981, "kl": 0.06188395326080354, "context": 0.0909440581308445}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.6488850116729736, "loss_scale": 4.0, "retries": 0, "seconds": 328.1, "max_memory_gb": 17.13, "ce": 0.06458359293173999, "kl": 0.04720035705395276, "context": 0.08818376761337277}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 2.3347387313842773, "loss_scale": 4.0, "retries": 0, "seconds": 329.6, "max_memory_gb": 17.13, "ce": 0.005448120231449138, "kl": 0.7402801977059426, "context": 0.3755882116674911}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 4.798404216766357, "loss_scale": 4.0, "retries": 0, "seconds": 331.0, "max_memory_gb": 17.13, "ce": 0.3232815740557271, "kl": 0.155416160763707, "context": 0.4009896552452119}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.14150065183639526, "loss_scale": 4.0, "retries": 0, "seconds": 332.6, "max_memory_gb": 17.13, "ce": 0.00798358433348767, "kl": 0.0006925054040038958, "context": 0.008329837073688395}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 3.293576955795288, "loss_scale": 4.0, "retries": 0, "seconds": 333.9, "max_memory_gb": 17.13, "ce": 0.3308272403664887, "kl": 0.2582893038634211, "context": 0.4599718973040581}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.45442071557044983, "loss_scale": 4.0, "retries": 0, "seconds": 335.5, "max_memory_gb": 17.13, "ce": 0.015083765611052513, "kl": 0.07598412611696403, "context": 0.05307582818204537}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 1.7037538290023804, "loss_scale": 4.0, "retries": 0, "seconds": 336.9, "max_memory_gb": 17.13, "ce": 0.2683160449414572, "kl": 0.12161560360073054, "context": 0.3291238495366997}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 2.0694754123687744, "loss_scale": 4.0, "retries": 0, "seconds": 338.4, "max_memory_gb": 17.13, "ce": 0.1817766105523333, "kl": 0.023865400082286214, "context": 0.19370931148296222}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 1.4650723934173584, "loss_scale": 4.0, "retries": 0, "seconds": 339.8, "max_memory_gb": 17.13, "ce": 0.034031493589282036, "kl": 0.11783957784928134, "context": 0.09295128157828003}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.8853714466094971, "loss_scale": 4.0, "retries": 0, "seconds": 341.4, "max_memory_gb": 17.13, "ce": 0.09590645568823675, "kl": 0.2581138243449459, "context": 0.22496337913798925}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 2.2403786182403564, "loss_scale": 4.0, "retries": 0, "seconds": 342.8, "max_memory_gb": 17.13, "ce": 0.03356223495211452, "kl": 0.14500902914005565, "context": 0.10606675152666867}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.06238817051053047, "loss_scale": 4.0, "retries": 0, "seconds": 344.3, "max_memory_gb": 17.13, "ce": 0.004043804714456201, "kl": 0.0012758323791786097, "context": 0.0046817209222353995}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.022933252155780792, "loss_scale": 4.0, "retries": 0, "seconds": 345.7, "max_memory_gb": 17.13, "ce": 0.001726464121020399, "kl": 0.00012828455146518536, "context": 0.0017906063731061295}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.4663406312465668, "loss_scale": 4.0, "retries": 0, "seconds": 347.2, "max_memory_gb": 17.13, "ce": 0.02367835865879897, "kl": 0.00798880540241953, "context": 0.027672762313159183}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.02326350472867489, "loss_scale": 4.0, "retries": 0, "seconds": 348.5, "max_memory_gb": 17.13, "ce": 0.0014532927743857726, "kl": 0.0007927224737613869, "context": 0.0018496540433261544}

runs/fixed_internvl/context/Full_s73/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Full_s73/adapter/adapter_config.json

@@ -0,0 +1,556 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.0.mlp.down_proj": 32,
+    "model.language_model.layers.0.mlp.gate_proj": 32,
+    "model.language_model.layers.0.mlp.up_proj": 32,
+    "model.language_model.layers.0.self_attn.k_proj": 32,
+    "model.language_model.layers.0.self_attn.o_proj": 32,
+    "model.language_model.layers.0.self_attn.q_proj": 32,
+    "model.language_model.layers.0.self_attn.v_proj": 32,
+    "model.language_model.layers.1.mlp.down_proj": 32,
+    "model.language_model.layers.1.mlp.gate_proj": 32,
+    "model.language_model.layers.1.mlp.up_proj": 32,
+    "model.language_model.layers.1.self_attn.k_proj": 32,
+    "model.language_model.layers.1.self_attn.o_proj": 32,
+    "model.language_model.layers.1.self_attn.q_proj": 32,
+    "model.language_model.layers.1.self_attn.v_proj": 32,
+    "model.language_model.layers.10.mlp.down_proj": 32,
+    "model.language_model.layers.10.mlp.gate_proj": 32,
+    "model.language_model.layers.10.mlp.up_proj": 32,
+    "model.language_model.layers.10.self_attn.k_proj": 32,
+    "model.language_model.layers.10.self_attn.o_proj": 32,
+    "model.language_model.layers.10.self_attn.q_proj": 32,
+    "model.language_model.layers.10.self_attn.v_proj": 32,
+    "model.language_model.layers.11.mlp.down_proj": 32,
+    "model.language_model.layers.11.mlp.gate_proj": 32,
+    "model.language_model.layers.11.mlp.up_proj": 32,
+    "model.language_model.layers.11.self_attn.k_proj": 32,
+    "model.language_model.layers.11.self_attn.o_proj": 32,
+    "model.language_model.layers.11.self_attn.q_proj": 32,
+    "model.language_model.layers.11.self_attn.v_proj": 32,
+    "model.language_model.layers.12.mlp.down_proj": 32,
+    "model.language_model.layers.12.mlp.gate_proj": 32,
+    "model.language_model.layers.12.mlp.up_proj": 32,
+    "model.language_model.layers.12.self_attn.k_proj": 32,
+    "model.language_model.layers.12.self_attn.o_proj": 32,
+    "model.language_model.layers.12.self_attn.q_proj": 32,
+    "model.language_model.layers.12.self_attn.v_proj": 32,
+    "model.language_model.layers.13.mlp.down_proj": 32,
+    "model.language_model.layers.13.mlp.gate_proj": 32,
+    "model.language_model.layers.13.mlp.up_proj": 32,
+    "model.language_model.layers.13.self_attn.k_proj": 32,
+    "model.language_model.layers.13.self_attn.o_proj": 32,
+    "model.language_model.layers.13.self_attn.q_proj": 32,
+    "model.language_model.layers.13.self_attn.v_proj": 32,
+    "model.language_model.layers.14.mlp.down_proj": 32,
+    "model.language_model.layers.14.mlp.gate_proj": 32,
+    "model.language_model.layers.14.mlp.up_proj": 32,
+    "model.language_model.layers.14.self_attn.k_proj": 32,
+    "model.language_model.layers.14.self_attn.o_proj": 32,
+    "model.language_model.layers.14.self_attn.q_proj": 32,
+    "model.language_model.layers.14.self_attn.v_proj": 32,
+    "model.language_model.layers.15.mlp.down_proj": 32,
+    "model.language_model.layers.15.mlp.gate_proj": 32,
+    "model.language_model.layers.15.mlp.up_proj": 32,
+    "model.language_model.layers.15.self_attn.k_proj": 32,
+    "model.language_model.layers.15.self_attn.o_proj": 32,
+    "model.language_model.layers.15.self_attn.q_proj": 32,
+    "model.language_model.layers.15.self_attn.v_proj": 32,
+    "model.language_model.layers.16.mlp.down_proj": 32,
+    "model.language_model.layers.16.mlp.gate_proj": 32,
+    "model.language_model.layers.16.mlp.up_proj": 32,
+    "model.language_model.layers.16.self_attn.k_proj": 32,
+    "model.language_model.layers.16.self_attn.o_proj": 32,
+    "model.language_model.layers.16.self_attn.q_proj": 32,
+    "model.language_model.layers.16.self_attn.v_proj": 32,
+    "model.language_model.layers.17.mlp.down_proj": 32,
+    "model.language_model.layers.17.mlp.gate_proj": 32,
+    "model.language_model.layers.17.mlp.up_proj": 32,
+    "model.language_model.layers.17.self_attn.k_proj": 32,
+    "model.language_model.layers.17.self_attn.o_proj": 32,
+    "model.language_model.layers.17.self_attn.q_proj": 32,
+    "model.language_model.layers.17.self_attn.v_proj": 32,
+    "model.language_model.layers.18.mlp.down_proj": 32,
+    "model.language_model.layers.18.mlp.gate_proj": 32,
+    "model.language_model.layers.18.mlp.up_proj": 32,
+    "model.language_model.layers.18.self_attn.k_proj": 32,
+    "model.language_model.layers.18.self_attn.o_proj": 32,
+    "model.language_model.layers.18.self_attn.q_proj": 32,
+    "model.language_model.layers.18.self_attn.v_proj": 32,
+    "model.language_model.layers.19.mlp.down_proj": 32,
+    "model.language_model.layers.19.mlp.gate_proj": 32,
+    "model.language_model.layers.19.mlp.up_proj": 32,
+    "model.language_model.layers.19.self_attn.k_proj": 32,
+    "model.language_model.layers.19.self_attn.o_proj": 32,
+    "model.language_model.layers.19.self_attn.q_proj": 32,
+    "model.language_model.layers.19.self_attn.v_proj": 32,
+    "model.language_model.layers.2.mlp.down_proj": 32,
+    "model.language_model.layers.2.mlp.gate_proj": 32,
+    "model.language_model.layers.2.mlp.up_proj": 32,
+    "model.language_model.layers.2.self_attn.k_proj": 32,
+    "model.language_model.layers.2.self_attn.o_proj": 32,
+    "model.language_model.layers.2.self_attn.q_proj": 32,
+    "model.language_model.layers.2.self_attn.v_proj": 32,
+    "model.language_model.layers.20.mlp.down_proj": 32,
+    "model.language_model.layers.20.mlp.gate_proj": 32,
+    "model.language_model.layers.20.mlp.up_proj": 32,
+    "model.language_model.layers.20.self_attn.k_proj": 32,
+    "model.language_model.layers.20.self_attn.o_proj": 32,
+    "model.language_model.layers.20.self_attn.q_proj": 32,
+    "model.language_model.layers.20.self_attn.v_proj": 32,
+    "model.language_model.layers.21.mlp.down_proj": 32,
+    "model.language_model.layers.21.mlp.gate_proj": 32,
+    "model.language_model.layers.21.mlp.up_proj": 32,
+    "model.language_model.layers.21.self_attn.k_proj": 32,
+    "model.language_model.layers.21.self_attn.o_proj": 32,
+    "model.language_model.layers.21.self_attn.q_proj": 32,
+    "model.language_model.layers.21.self_attn.v_proj": 32,
+    "model.language_model.layers.22.mlp.down_proj": 32,
+    "model.language_model.layers.22.mlp.gate_proj": 32,
+    "model.language_model.layers.22.mlp.up_proj": 32,
+    "model.language_model.layers.22.self_attn.k_proj": 32,
+    "model.language_model.layers.22.self_attn.o_proj": 32,
+    "model.language_model.layers.22.self_attn.q_proj": 32,
+    "model.language_model.layers.22.self_attn.v_proj": 32,
+    "model.language_model.layers.23.mlp.down_proj": 32,
+    "model.language_model.layers.23.mlp.gate_proj": 32,
+    "model.language_model.layers.23.mlp.up_proj": 32,
+    "model.language_model.layers.23.self_attn.k_proj": 32,
+    "model.language_model.layers.23.self_attn.o_proj": 32,
+    "model.language_model.layers.23.self_attn.q_proj": 32,
+    "model.language_model.layers.23.self_attn.v_proj": 32,
+    "model.language_model.layers.24.mlp.down_proj": 32,
+    "model.language_model.layers.24.mlp.gate_proj": 32,
+    "model.language_model.layers.24.mlp.up_proj": 32,
+    "model.language_model.layers.24.self_attn.k_proj": 32,
+    "model.language_model.layers.24.self_attn.o_proj": 32,
+    "model.language_model.layers.24.self_attn.q_proj": 32,
+    "model.language_model.layers.24.self_attn.v_proj": 32,
+    "model.language_model.layers.25.mlp.down_proj": 32,
+    "model.language_model.layers.25.mlp.gate_proj": 32,
+    "model.language_model.layers.25.mlp.up_proj": 32,
+    "model.language_model.layers.25.self_attn.k_proj": 32,
+    "model.language_model.layers.25.self_attn.o_proj": 32,
+    "model.language_model.layers.25.self_attn.q_proj": 32,
+    "model.language_model.layers.25.self_attn.v_proj": 32,
+    "model.language_model.layers.26.mlp.down_proj": 32,
+    "model.language_model.layers.26.mlp.gate_proj": 32,
+    "model.language_model.layers.26.mlp.up_proj": 32,
+    "model.language_model.layers.26.self_attn.k_proj": 32,
+    "model.language_model.layers.26.self_attn.o_proj": 32,
+    "model.language_model.layers.26.self_attn.q_proj": 32,
+    "model.language_model.layers.26.self_attn.v_proj": 32,
+    "model.language_model.layers.27.mlp.down_proj": 32,
+    "model.language_model.layers.27.mlp.gate_proj": 32,
+    "model.language_model.layers.27.mlp.up_proj": 32,
+    "model.language_model.layers.27.self_attn.k_proj": 32,
+    "model.language_model.layers.27.self_attn.o_proj": 32,
+    "model.language_model.layers.27.self_attn.q_proj": 32,
+    "model.language_model.layers.27.self_attn.v_proj": 32,
+    "model.language_model.layers.28.mlp.down_proj": 32,
+    "model.language_model.layers.28.mlp.gate_proj": 32,
+    "model.language_model.layers.28.mlp.up_proj": 32,
+    "model.language_model.layers.28.self_attn.k_proj": 32,
+    "model.language_model.layers.28.self_attn.o_proj": 32,
+    "model.language_model.layers.28.self_attn.q_proj": 32,
+    "model.language_model.layers.28.self_attn.v_proj": 32,
+    "model.language_model.layers.29.mlp.down_proj": 32,
+    "model.language_model.layers.29.mlp.gate_proj": 32,
+    "model.language_model.layers.29.mlp.up_proj": 32,
+    "model.language_model.layers.29.self_attn.k_proj": 32,
+    "model.language_model.layers.29.self_attn.o_proj": 32,
+    "model.language_model.layers.29.self_attn.q_proj": 32,
+    "model.language_model.layers.29.self_attn.v_proj": 32,
+    "model.language_model.layers.3.mlp.down_proj": 32,
+    "model.language_model.layers.3.mlp.gate_proj": 32,
+    "model.language_model.layers.3.mlp.up_proj": 32,
+    "model.language_model.layers.3.self_attn.k_proj": 32,
+    "model.language_model.layers.3.self_attn.o_proj": 32,
+    "model.language_model.layers.3.self_attn.q_proj": 32,
+    "model.language_model.layers.3.self_attn.v_proj": 32,
+    "model.language_model.layers.30.mlp.down_proj": 32,
+    "model.language_model.layers.30.mlp.gate_proj": 32,
+    "model.language_model.layers.30.mlp.up_proj": 32,
+    "model.language_model.layers.30.self_attn.k_proj": 32,
+    "model.language_model.layers.30.self_attn.o_proj": 32,
+    "model.language_model.layers.30.self_attn.q_proj": 32,
+    "model.language_model.layers.30.self_attn.v_proj": 32,
+    "model.language_model.layers.31.mlp.down_proj": 32,
+    "model.language_model.layers.31.mlp.gate_proj": 32,
+    "model.language_model.layers.31.mlp.up_proj": 32,
+    "model.language_model.layers.31.self_attn.k_proj": 32,
+    "model.language_model.layers.31.self_attn.o_proj": 32,
+    "model.language_model.layers.31.self_attn.q_proj": 32,
+    "model.language_model.layers.31.self_attn.v_proj": 32,
+    "model.language_model.layers.32.mlp.down_proj": 32,
+    "model.language_model.layers.32.mlp.gate_proj": 32,
+    "model.language_model.layers.32.mlp.up_proj": 32,
+    "model.language_model.layers.32.self_attn.k_proj": 32,
+    "model.language_model.layers.32.self_attn.o_proj": 32,
+    "model.language_model.layers.32.self_attn.q_proj": 32,
+    "model.language_model.layers.32.self_attn.v_proj": 32,
+    "model.language_model.layers.33.mlp.down_proj": 32,
+    "model.language_model.layers.33.mlp.gate_proj": 32,
+    "model.language_model.layers.33.mlp.up_proj": 32,
+    "model.language_model.layers.33.self_attn.k_proj": 32,
+    "model.language_model.layers.33.self_attn.o_proj": 32,
+    "model.language_model.layers.33.self_attn.q_proj": 32,
+    "model.language_model.layers.33.self_attn.v_proj": 32,
+    "model.language_model.layers.34.mlp.down_proj": 32,
+    "model.language_model.layers.34.mlp.gate_proj": 32,
+    "model.language_model.layers.34.mlp.up_proj": 32,
+    "model.language_model.layers.34.self_attn.k_proj": 32,
+    "model.language_model.layers.34.self_attn.o_proj": 32,
+    "model.language_model.layers.34.self_attn.q_proj": 32,
+    "model.language_model.layers.34.self_attn.v_proj": 32,
+    "model.language_model.layers.35.mlp.down_proj": 32,
+    "model.language_model.layers.35.mlp.gate_proj": 32,
+    "model.language_model.layers.35.mlp.up_proj": 32,
+    "model.language_model.layers.35.self_attn.k_proj": 32,
+    "model.language_model.layers.35.self_attn.o_proj": 32,
+    "model.language_model.layers.35.self_attn.q_proj": 32,
+    "model.language_model.layers.35.self_attn.v_proj": 32,
+    "model.language_model.layers.4.mlp.down_proj": 32,
+    "model.language_model.layers.4.mlp.gate_proj": 32,
+    "model.language_model.layers.4.mlp.up_proj": 32,
+    "model.language_model.layers.4.self_attn.k_proj": 32,
+    "model.language_model.layers.4.self_attn.o_proj": 32,
+    "model.language_model.layers.4.self_attn.q_proj": 32,
+    "model.language_model.layers.4.self_attn.v_proj": 32,
+    "model.language_model.layers.5.mlp.down_proj": 32,
+    "model.language_model.layers.5.mlp.gate_proj": 32,
+    "model.language_model.layers.5.mlp.up_proj": 32,
+    "model.language_model.layers.5.self_attn.k_proj": 32,
+    "model.language_model.layers.5.self_attn.o_proj": 32,
+    "model.language_model.layers.5.self_attn.q_proj": 32,
+    "model.language_model.layers.5.self_attn.v_proj": 32,
+    "model.language_model.layers.6.mlp.down_proj": 32,
+    "model.language_model.layers.6.mlp.gate_proj": 32,
+    "model.language_model.layers.6.mlp.up_proj": 32,
+    "model.language_model.layers.6.self_attn.k_proj": 32,
+    "model.language_model.layers.6.self_attn.o_proj": 32,
+    "model.language_model.layers.6.self_attn.q_proj": 32,
+    "model.language_model.layers.6.self_attn.v_proj": 32,
+    "model.language_model.layers.7.mlp.down_proj": 32,
+    "model.language_model.layers.7.mlp.gate_proj": 32,
+    "model.language_model.layers.7.mlp.up_proj": 32,
+    "model.language_model.layers.7.self_attn.k_proj": 32,
+    "model.language_model.layers.7.self_attn.o_proj": 32,
+    "model.language_model.layers.7.self_attn.q_proj": 32,
+    "model.language_model.layers.7.self_attn.v_proj": 32,
+    "model.language_model.layers.8.mlp.down_proj": 32,
+    "model.language_model.layers.8.mlp.gate_proj": 32,
+    "model.language_model.layers.8.mlp.up_proj": 32,
+    "model.language_model.layers.8.self_attn.k_proj": 32,
+    "model.language_model.layers.8.self_attn.o_proj": 32,
+    "model.language_model.layers.8.self_attn.q_proj": 32,
+    "model.language_model.layers.8.self_attn.v_proj": 32,
+    "model.language_model.layers.9.mlp.down_proj": 32,
+    "model.language_model.layers.9.mlp.gate_proj": 32,
+    "model.language_model.layers.9.mlp.up_proj": 32,
+    "model.language_model.layers.9.self_attn.k_proj": 32,
+    "model.language_model.layers.9.self_attn.o_proj": 32,
+    "model.language_model.layers.9.self_attn.q_proj": 32,
+    "model.language_model.layers.9.self_attn.v_proj": 32
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.0.mlp.down_proj": 16,
+    "model.language_model.layers.0.mlp.gate_proj": 16,
+    "model.language_model.layers.0.mlp.up_proj": 16,
+    "model.language_model.layers.0.self_attn.k_proj": 16,
+    "model.language_model.layers.0.self_attn.o_proj": 16,
+    "model.language_model.layers.0.self_attn.q_proj": 16,
+    "model.language_model.layers.0.self_attn.v_proj": 16,
+    "model.language_model.layers.1.mlp.down_proj": 16,
+    "model.language_model.layers.1.mlp.gate_proj": 16,
+    "model.language_model.layers.1.mlp.up_proj": 16,
+    "model.language_model.layers.1.self_attn.k_proj": 16,
+    "model.language_model.layers.1.self_attn.o_proj": 16,
+    "model.language_model.layers.1.self_attn.q_proj": 16,
+    "model.language_model.layers.1.self_attn.v_proj": 16,
+    "model.language_model.layers.10.mlp.down_proj": 16,
+    "model.language_model.layers.10.mlp.gate_proj": 16,
+    "model.language_model.layers.10.mlp.up_proj": 16,
+    "model.language_model.layers.10.self_attn.k_proj": 16,
+    "model.language_model.layers.10.self_attn.o_proj": 16,
+    "model.language_model.layers.10.self_attn.q_proj": 16,
+    "model.language_model.layers.10.self_attn.v_proj": 16,
+    "model.language_model.layers.11.mlp.down_proj": 16,
+    "model.language_model.layers.11.mlp.gate_proj": 16,
+    "model.language_model.layers.11.mlp.up_proj": 16,
+    "model.language_model.layers.11.self_attn.k_proj": 16,
+    "model.language_model.layers.11.self_attn.o_proj": 16,
+    "model.language_model.layers.11.self_attn.q_proj": 16,
+    "model.language_model.layers.11.self_attn.v_proj": 16,
+    "model.language_model.layers.12.mlp.down_proj": 16,
+    "model.language_model.layers.12.mlp.gate_proj": 16,
+    "model.language_model.layers.12.mlp.up_proj": 16,
+    "model.language_model.layers.12.self_attn.k_proj": 16,
+    "model.language_model.layers.12.self_attn.o_proj": 16,
+    "model.language_model.layers.12.self_attn.q_proj": 16,
+    "model.language_model.layers.12.self_attn.v_proj": 16,
+    "model.language_model.layers.13.mlp.down_proj": 16,
+    "model.language_model.layers.13.mlp.gate_proj": 16,
+    "model.language_model.layers.13.mlp.up_proj": 16,
+    "model.language_model.layers.13.self_attn.k_proj": 16,
+    "model.language_model.layers.13.self_attn.o_proj": 16,
+    "model.language_model.layers.13.self_attn.q_proj": 16,
+    "model.language_model.layers.13.self_attn.v_proj": 16,
+    "model.language_model.layers.14.mlp.down_proj": 16,
+    "model.language_model.layers.14.mlp.gate_proj": 16,
+    "model.language_model.layers.14.mlp.up_proj": 16,
+    "model.language_model.layers.14.self_attn.k_proj": 16,
+    "model.language_model.layers.14.self_attn.o_proj": 16,
+    "model.language_model.layers.14.self_attn.q_proj": 16,
+    "model.language_model.layers.14.self_attn.v_proj": 16,
+    "model.language_model.layers.15.mlp.down_proj": 16,
+    "model.language_model.layers.15.mlp.gate_proj": 16,
+    "model.language_model.layers.15.mlp.up_proj": 16,
+    "model.language_model.layers.15.self_attn.k_proj": 16,
+    "model.language_model.layers.15.self_attn.o_proj": 16,
+    "model.language_model.layers.15.self_attn.q_proj": 16,
+    "model.language_model.layers.15.self_attn.v_proj": 16,
+    "model.language_model.layers.16.mlp.down_proj": 16,
+    "model.language_model.layers.16.mlp.gate_proj": 16,
+    "model.language_model.layers.16.mlp.up_proj": 16,
+    "model.language_model.layers.16.self_attn.k_proj": 16,
+    "model.language_model.layers.16.self_attn.o_proj": 16,
+    "model.language_model.layers.16.self_attn.q_proj": 16,
+    "model.language_model.layers.16.self_attn.v_proj": 16,
+    "model.language_model.layers.17.mlp.down_proj": 16,
+    "model.language_model.layers.17.mlp.gate_proj": 16,
+    "model.language_model.layers.17.mlp.up_proj": 16,
+    "model.language_model.layers.17.self_attn.k_proj": 16,
+    "model.language_model.layers.17.self_attn.o_proj": 16,
+    "model.language_model.layers.17.self_attn.q_proj": 16,
+    "model.language_model.layers.17.self_attn.v_proj": 16,
+    "model.language_model.layers.18.mlp.down_proj": 16,
+    "model.language_model.layers.18.mlp.gate_proj": 16,
+    "model.language_model.layers.18.mlp.up_proj": 16,
+    "model.language_model.layers.18.self_attn.k_proj": 16,
+    "model.language_model.layers.18.self_attn.o_proj": 16,
+    "model.language_model.layers.18.self_attn.q_proj": 16,
+    "model.language_model.layers.18.self_attn.v_proj": 16,
+    "model.language_model.layers.19.mlp.down_proj": 16,
+    "model.language_model.layers.19.mlp.gate_proj": 16,
+    "model.language_model.layers.19.mlp.up_proj": 16,
+    "model.language_model.layers.19.self_attn.k_proj": 16,
+    "model.language_model.layers.19.self_attn.o_proj": 16,
+    "model.language_model.layers.19.self_attn.q_proj": 16,
+    "model.language_model.layers.19.self_attn.v_proj": 16,
+    "model.language_model.layers.2.mlp.down_proj": 16,
+    "model.language_model.layers.2.mlp.gate_proj": 16,
+    "model.language_model.layers.2.mlp.up_proj": 16,
+    "model.language_model.layers.2.self_attn.k_proj": 16,
+    "model.language_model.layers.2.self_attn.o_proj": 16,
+    "model.language_model.layers.2.self_attn.q_proj": 16,
+    "model.language_model.layers.2.self_attn.v_proj": 16,
+    "model.language_model.layers.20.mlp.down_proj": 16,
+    "model.language_model.layers.20.mlp.gate_proj": 16,
+    "model.language_model.layers.20.mlp.up_proj": 16,
+    "model.language_model.layers.20.self_attn.k_proj": 16,
+    "model.language_model.layers.20.self_attn.o_proj": 16,
+    "model.language_model.layers.20.self_attn.q_proj": 16,
+    "model.language_model.layers.20.self_attn.v_proj": 16,
+    "model.language_model.layers.21.mlp.down_proj": 16,
+    "model.language_model.layers.21.mlp.gate_proj": 16,
+    "model.language_model.layers.21.mlp.up_proj": 16,
+    "model.language_model.layers.21.self_attn.k_proj": 16,
+    "model.language_model.layers.21.self_attn.o_proj": 16,
+    "model.language_model.layers.21.self_attn.q_proj": 16,
+    "model.language_model.layers.21.self_attn.v_proj": 16,
+    "model.language_model.layers.22.mlp.down_proj": 16,
+    "model.language_model.layers.22.mlp.gate_proj": 16,
+    "model.language_model.layers.22.mlp.up_proj": 16,
+    "model.language_model.layers.22.self_attn.k_proj": 16,
+    "model.language_model.layers.22.self_attn.o_proj": 16,
+    "model.language_model.layers.22.self_attn.q_proj": 16,
+    "model.language_model.layers.22.self_attn.v_proj": 16,
+    "model.language_model.layers.23.mlp.down_proj": 16,
+    "model.language_model.layers.23.mlp.gate_proj": 16,
+    "model.language_model.layers.23.mlp.up_proj": 16,
+    "model.language_model.layers.23.self_attn.k_proj": 16,
+    "model.language_model.layers.23.self_attn.o_proj": 16,
+    "model.language_model.layers.23.self_attn.q_proj": 16,
+    "model.language_model.layers.23.self_attn.v_proj": 16,
+    "model.language_model.layers.24.mlp.down_proj": 16,
+    "model.language_model.layers.24.mlp.gate_proj": 16,
+    "model.language_model.layers.24.mlp.up_proj": 16,
+    "model.language_model.layers.24.self_attn.k_proj": 16,
+    "model.language_model.layers.24.self_attn.o_proj": 16,
+    "model.language_model.layers.24.self_attn.q_proj": 16,
+    "model.language_model.layers.24.self_attn.v_proj": 16,
+    "model.language_model.layers.25.mlp.down_proj": 16,
+    "model.language_model.layers.25.mlp.gate_proj": 16,
+    "model.language_model.layers.25.mlp.up_proj": 16,
+    "model.language_model.layers.25.self_attn.k_proj": 16,
+    "model.language_model.layers.25.self_attn.o_proj": 16,
+    "model.language_model.layers.25.self_attn.q_proj": 16,
+    "model.language_model.layers.25.self_attn.v_proj": 16,
+    "model.language_model.layers.26.mlp.down_proj": 16,
+    "model.language_model.layers.26.mlp.gate_proj": 16,
+    "model.language_model.layers.26.mlp.up_proj": 16,
+    "model.language_model.layers.26.self_attn.k_proj": 16,
+    "model.language_model.layers.26.self_attn.o_proj": 16,
+    "model.language_model.layers.26.self_attn.q_proj": 16,
+    "model.language_model.layers.26.self_attn.v_proj": 16,
+    "model.language_model.layers.27.mlp.down_proj": 16,
+    "model.language_model.layers.27.mlp.gate_proj": 16,
+    "model.language_model.layers.27.mlp.up_proj": 16,
+    "model.language_model.layers.27.self_attn.k_proj": 16,
+    "model.language_model.layers.27.self_attn.o_proj": 16,
+    "model.language_model.layers.27.self_attn.q_proj": 16,
+    "model.language_model.layers.27.self_attn.v_proj": 16,
+    "model.language_model.layers.28.mlp.down_proj": 16,
+    "model.language_model.layers.28.mlp.gate_proj": 16,
+    "model.language_model.layers.28.mlp.up_proj": 16,
+    "model.language_model.layers.28.self_attn.k_proj": 16,
+    "model.language_model.layers.28.self_attn.o_proj": 16,
+    "model.language_model.layers.28.self_attn.q_proj": 16,
+    "model.language_model.layers.28.self_attn.v_proj": 16,
+    "model.language_model.layers.29.mlp.down_proj": 16,
+    "model.language_model.layers.29.mlp.gate_proj": 16,
+    "model.language_model.layers.29.mlp.up_proj": 16,
+    "model.language_model.layers.29.self_attn.k_proj": 16,
+    "model.language_model.layers.29.self_attn.o_proj": 16,
+    "model.language_model.layers.29.self_attn.q_proj": 16,
+    "model.language_model.layers.29.self_attn.v_proj": 16,
+    "model.language_model.layers.3.mlp.down_proj": 16,
+    "model.language_model.layers.3.mlp.gate_proj": 16,
+    "model.language_model.layers.3.mlp.up_proj": 16,
+    "model.language_model.layers.3.self_attn.k_proj": 16,
+    "model.language_model.layers.3.self_attn.o_proj": 16,
+    "model.language_model.layers.3.self_attn.q_proj": 16,
+    "model.language_model.layers.3.self_attn.v_proj": 16,
+    "model.language_model.layers.30.mlp.down_proj": 16,
+    "model.language_model.layers.30.mlp.gate_proj": 16,
+    "model.language_model.layers.30.mlp.up_proj": 16,
+    "model.language_model.layers.30.self_attn.k_proj": 16,
+    "model.language_model.layers.30.self_attn.o_proj": 16,
+    "model.language_model.layers.30.self_attn.q_proj": 16,
+    "model.language_model.layers.30.self_attn.v_proj": 16,
+    "model.language_model.layers.31.mlp.down_proj": 16,
+    "model.language_model.layers.31.mlp.gate_proj": 16,
+    "model.language_model.layers.31.mlp.up_proj": 16,
+    "model.language_model.layers.31.self_attn.k_proj": 16,
+    "model.language_model.layers.31.self_attn.o_proj": 16,
+    "model.language_model.layers.31.self_attn.q_proj": 16,
+    "model.language_model.layers.31.self_attn.v_proj": 16,
+    "model.language_model.layers.32.mlp.down_proj": 16,
+    "model.language_model.layers.32.mlp.gate_proj": 16,
+    "model.language_model.layers.32.mlp.up_proj": 16,
+    "model.language_model.layers.32.self_attn.k_proj": 16,
+    "model.language_model.layers.32.self_attn.o_proj": 16,
+    "model.language_model.layers.32.self_attn.q_proj": 16,
+    "model.language_model.layers.32.self_attn.v_proj": 16,
+    "model.language_model.layers.33.mlp.down_proj": 16,
+    "model.language_model.layers.33.mlp.gate_proj": 16,
+    "model.language_model.layers.33.mlp.up_proj": 16,
+    "model.language_model.layers.33.self_attn.k_proj": 16,
+    "model.language_model.layers.33.self_attn.o_proj": 16,
+    "model.language_model.layers.33.self_attn.q_proj": 16,
+    "model.language_model.layers.33.self_attn.v_proj": 16,
+    "model.language_model.layers.34.mlp.down_proj": 16,
+    "model.language_model.layers.34.mlp.gate_proj": 16,
+    "model.language_model.layers.34.mlp.up_proj": 16,
+    "model.language_model.layers.34.self_attn.k_proj": 16,
+    "model.language_model.layers.34.self_attn.o_proj": 16,
+    "model.language_model.layers.34.self_attn.q_proj": 16,
+    "model.language_model.layers.34.self_attn.v_proj": 16,
+    "model.language_model.layers.35.mlp.down_proj": 16,
+    "model.language_model.layers.35.mlp.gate_proj": 16,
+    "model.language_model.layers.35.mlp.up_proj": 16,
+    "model.language_model.layers.35.self_attn.k_proj": 16,
+    "model.language_model.layers.35.self_attn.o_proj": 16,
+    "model.language_model.layers.35.self_attn.q_proj": 16,
+    "model.language_model.layers.35.self_attn.v_proj": 16,
+    "model.language_model.layers.4.mlp.down_proj": 16,
+    "model.language_model.layers.4.mlp.gate_proj": 16,
+    "model.language_model.layers.4.mlp.up_proj": 16,
+    "model.language_model.layers.4.self_attn.k_proj": 16,
+    "model.language_model.layers.4.self_attn.o_proj": 16,
+    "model.language_model.layers.4.self_attn.q_proj": 16,
+    "model.language_model.layers.4.self_attn.v_proj": 16,
+    "model.language_model.layers.5.mlp.down_proj": 16,
+    "model.language_model.layers.5.mlp.gate_proj": 16,
+    "model.language_model.layers.5.mlp.up_proj": 16,
+    "model.language_model.layers.5.self_attn.k_proj": 16,
+    "model.language_model.layers.5.self_attn.o_proj": 16,
+    "model.language_model.layers.5.self_attn.q_proj": 16,
+    "model.language_model.layers.5.self_attn.v_proj": 16,
+    "model.language_model.layers.6.mlp.down_proj": 16,
+    "model.language_model.layers.6.mlp.gate_proj": 16,
+    "model.language_model.layers.6.mlp.up_proj": 16,
+    "model.language_model.layers.6.self_attn.k_proj": 16,
+    "model.language_model.layers.6.self_attn.o_proj": 16,
+    "model.language_model.layers.6.self_attn.q_proj": 16,
+    "model.language_model.layers.6.self_attn.v_proj": 16,
+    "model.language_model.layers.7.mlp.down_proj": 16,
+    "model.language_model.layers.7.mlp.gate_proj": 16,
+    "model.language_model.layers.7.mlp.up_proj": 16,
+    "model.language_model.layers.7.self_attn.k_proj": 16,
+    "model.language_model.layers.7.self_attn.o_proj": 16,
+    "model.language_model.layers.7.self_attn.q_proj": 16,
+    "model.language_model.layers.7.self_attn.v_proj": 16,
+    "model.language_model.layers.8.mlp.down_proj": 16,
+    "model.language_model.layers.8.mlp.gate_proj": 16,
+    "model.language_model.layers.8.mlp.up_proj": 16,
+    "model.language_model.layers.8.self_attn.k_proj": 16,
+    "model.language_model.layers.8.self_attn.o_proj": 16,
+    "model.language_model.layers.8.self_attn.q_proj": 16,
+    "model.language_model.layers.8.self_attn.v_proj": 16,
+    "model.language_model.layers.9.mlp.down_proj": 16,
+    "model.language_model.layers.9.mlp.gate_proj": 16,
+    "model.language_model.layers.9.mlp.up_proj": 16,
+    "model.language_model.layers.9.self_attn.k_proj": 16,
+    "model.language_model.layers.9.self_attn.o_proj": 16,
+    "model.language_model.layers.9.self_attn.q_proj": 16,
+    "model.language_model.layers.9.self_attn.v_proj": 16
+  },
+  "revision": null,
+  "target_modules": [
+    "self_attn.k_proj",
+    "self_attn.q_proj",
+    "up_proj",
+    "self_attn.v_proj",
+    "down_proj",
+    "o_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Full_s73/config.json

@@ -0,0 +1,99 @@
+{
+  "behavior": "context",
+  "seed": 73,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Full",
+    "layers": [
+      0,
+      1,
+      2,
+      3,
+      4,
+      5,
+      6,
+      7,
+      8,
+      9,
+      10,
+      11,
+      12,
+      13,
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21,
+      22,
+      23,
+      24,
+      25,
+      26,
+      27,
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35
+    ],
+    "ranks": {
+      "0": 16,
+      "1": 16,
+      "2": 16,
+      "3": 16,
+      "4": 16,
+      "5": 16,
+      "6": 16,
+      "7": 16,
+      "8": 16,
+      "9": 16,
+      "10": 16,
+      "11": 16,
+      "12": 16,
+      "13": 16,
+      "14": 16,
+      "15": 16,
+      "16": 16,
+      "17": 16,
+      "18": 16,
+      "19": 16,
+      "20": 16,
+      "21": 16,
+      "22": 16,
+      "23": 16,
+      "24": 16,
+      "25": 16,
+      "26": 16,
+      "27": 16,
+      "28": 16,
+      "29": 16,
+      "30": 16,
+      "31": 16,
+      "32": 16,
+      "33": 16,
+      "34": 16,
+      "35": 16
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Full_s73/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 12.873872756958008, "loss_scale": 4.0, "retries": 0, "seconds": 2.9, "max_memory_gb": 16.8, "ce": 0.6797246597125195, "kl": 0.0438283761832281, "context": 0.7016388622396335}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 22.35909652709961, "loss_scale": 4.0, "retries": 0, "seconds": 4.6, "max_memory_gb": 17.12, "ce": 1.7120159268379211, "kl": 0.6940609631128609, "context": 2.0590464174747467}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 10.83067798614502, "loss_scale": 4.0, "retries": 0, "seconds": 6.4, "max_memory_gb": 17.12, "ce": 0.5760454892879352, "kl": 0.10990261868573725, "context": 0.6309968116693199}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 12.084782600402832, "loss_scale": 4.0, "retries": 0, "seconds": 8.1, "max_memory_gb": 17.12, "ce": 0.6178208285127766, "kl": 0.018383910854026908, "context": 0.6270127967582084}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 5.794358253479004, "loss_scale": 4.0, "retries": 0, "seconds": 9.7, "max_memory_gb": 17.12, "ce": 0.25101549886858265, "kl": 0.1270263169426471, "context": 0.3145286627113819}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 3.670262575149536, "loss_scale": 4.0, "retries": 0, "seconds": 11.2, "max_memory_gb": 17.12, "ce": 0.25186506286263466, "kl": 0.05658796662464738, "context": 0.2801590543240309}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 0.4406190514564514, "loss_scale": 4.0, "retries": 0, "seconds": 12.8, "max_memory_gb": 17.12, "ce": 0.023320833835896337, "kl": 0.012946363445735187, "context": 0.029794015965308063}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.7364715337753296, "loss_scale": 4.0, "retries": 0, "seconds": 14.2, "max_memory_gb": 17.12, "ce": 0.3475821052561514, "kl": 0.5009108946978813, "context": 0.5980375577346422}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 2.4197819232940674, "loss_scale": 4.0, "retries": 0, "seconds": 15.8, "max_memory_gb": 17.12, "ce": 0.04647810838650912, "kl": 0.5006608025287278, "context": 0.2968085128813982}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 1.5828688144683838, "loss_scale": 4.0, "retries": 0, "seconds": 17.2, "max_memory_gb": 17.12, "ce": 0.03892153801280074, "kl": 0.36589281337046486, "context": 0.22186794935259968}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.5159640312194824, "loss_scale": 4.0, "retries": 0, "seconds": 19.0, "max_memory_gb": 17.12, "ce": 0.013721237424761057, "kl": 0.09023927082307637, "context": 0.05884087341837585}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 1.6483631134033203, "loss_scale": 4.0, "retries": 0, "seconds": 20.4, "max_memory_gb": 17.12, "ce": 0.029169965644769036, "kl": 0.024743433241383173, "context": 0.041541683371178806}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 2.509115695953369, "loss_scale": 4.0, "retries": 0, "seconds": 22.0, "max_memory_gb": 17.12, "ce": 0.15168079865543405, "kl": 0.08267153504129965, "context": 0.1930165736703202}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 0.7261970639228821, "loss_scale": 4.0, "retries": 0, "seconds": 23.4, "max_memory_gb": 17.12, "ce": 0.04546169252716936, "kl": 0.08669273999112193, "context": 0.08880806635715999}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 10.064582824707031, "loss_scale": 4.0, "retries": 0, "seconds": 25.0, "max_memory_gb": 17.12, "ce": 0.13213117627310567, "kl": 0.16781365330098197, "context": 0.21603799553122371}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 4.519026756286621, "loss_scale": 4.0, "retries": 0, "seconds": 26.4, "max_memory_gb": 17.12, "ce": 0.7985457675968064, "kl": 0.09741681699233595, "context": 0.8472541531082243}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 3.6628634929656982, "loss_scale": 4.0, "retries": 0, "seconds": 28.0, "max_memory_gb": 17.12, "ce": 0.06414590665372089, "kl": 0.38057685541934916, "context": 0.25443432899191976}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 2.0312206745147705, "loss_scale": 4.0, "retries": 0, "seconds": 29.4, "max_memory_gb": 17.12, "ce": 0.0019188460137229413, "kl": 0.27172958997107344, "context": 0.13778363415622152}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 1.73723566532135, "loss_scale": 4.0, "retries": 0, "seconds": 31.0, "max_memory_gb": 17.12, "ce": 0.25476005458040163, "kl": 0.23410182870134122, "context": 0.37181097288703313}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 1.3599227666854858, "loss_scale": 4.0, "retries": 0, "seconds": 32.5, "max_memory_gb": 17.12, "ce": 0.07908981007130933, "kl": 0.33217284198326524, "context": 0.2451762385026086}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.1200568899512291, "loss_scale": 4.0, "retries": 0, "seconds": 34.0, "max_memory_gb": 17.12, "ce": 0.007498398728785105, "kl": 0.0009892876350932056, "context": 0.007993042716407217}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 1.0739495754241943, "loss_scale": 4.0, "retries": 0, "seconds": 35.5, "max_memory_gb": 17.12, "ce": 0.16742614420945756, "kl": 0.24671507003949955, "context": 0.29078366979956627}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 2.3147706985473633, "loss_scale": 4.0, "retries": 0, "seconds": 36.9, "max_memory_gb": 17.12, "ce": 0.15900052986398805, "kl": 0.10715980653912993, "context": 0.21258042940462474}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 1.9511218070983887, "loss_scale": 4.0, "retries": 0, "seconds": 38.5, "max_memory_gb": 17.12, "ce": 0.09177681169239804, "kl": 0.27499566002370557, "context": 0.22927465237444267}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 1.2016143798828125, "loss_scale": 4.0, "retries": 0, "seconds": 40.0, "max_memory_gb": 17.12, "ce": 0.056904952274635434, "kl": 0.04698881607328076, "context": 0.08039935817942023}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 5.096212863922119, "loss_scale": 4.0, "retries": 0, "seconds": 41.5, "max_memory_gb": 17.12, "ce": 0.18024393756058998, "kl": 0.41971561479476804, "context": 0.39010176172450883}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 7.63439416885376, "loss_scale": 4.0, "retries": 0, "seconds": 43.1, "max_memory_gb": 17.12, "ce": 0.16825181061722105, "kl": 0.22742165362797095, "context": 0.28196265210863203}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 0.052227187901735306, "loss_scale": 4.0, "retries": 0, "seconds": 44.5, "max_memory_gb": 17.12, "ce": 0.002568587544374168, "kl": 0.0005065037948952522, "context": 0.0028218393854331225}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 0.2220090627670288, "loss_scale": 4.0, "retries": 0, "seconds": 45.9, "max_memory_gb": 17.12, "ce": 0.009804117027670145, "kl": 0.0073093915125355124, "context": 0.013458812842145562}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 2.59747314453125, "loss_scale": 4.0, "retries": 0, "seconds": 47.5, "max_memory_gb": 17.12, "ce": 0.04271199647337198, "kl": 0.06523629176081158, "context": 0.07533014239743352}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 1.9038898944854736, "loss_scale": 4.0, "retries": 0, "seconds": 48.9, "max_memory_gb": 17.12, "ce": 0.008768253959715366, "kl": 0.5661046130117029, "context": 0.29182055982528254}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 5.863399505615234, "loss_scale": 4.0, "retries": 0, "seconds": 50.5, "max_memory_gb": 17.12, "ce": 0.18407758546527475, "kl": 0.511036862822948, "context": 0.4395960320252925}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.35827213525772095, "loss_scale": 4.0, "retries": 0, "seconds": 51.9, "max_memory_gb": 17.12, "ce": 0.01708588255860377, "kl": 0.031267327853129245, "context": 0.03271954576484859}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.7213606238365173, "loss_scale": 4.0, "retries": 0, "seconds": 53.4, "max_memory_gb": 17.12, "ce": 0.018065174779621884, "kl": 0.305377362674335, "context": 0.17075385796488263}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 1.6447343826293945, "loss_scale": 4.0, "retries": 0, "seconds": 54.9, "max_memory_gb": 17.12, "ce": 0.030586711538489908, "kl": 0.14506870033437735, "context": 0.10312106320634484}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 0.8659040331840515, "loss_scale": 4.0, "retries": 0, "seconds": 56.5, "max_memory_gb": 17.13, "ce": 0.06785548955667764, "kl": 0.029561207673395984, "context": 0.08263609564164653}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 0.27310502529144287, "loss_scale": 4.0, "retries": 0, "seconds": 57.9, "max_memory_gb": 17.13, "ce": 0.01092250901274383, "kl": 0.01599377836100757, "context": 0.01891939842607826}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.3787302076816559, "loss_scale": 4.0, "retries": 0, "seconds": 59.5, "max_memory_gb": 17.13, "ce": 0.030080965399974957, "kl": 0.017876316002002568, "context": 0.03901912522269413}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.970679521560669, "loss_scale": 4.0, "retries": 0, "seconds": 60.9, "max_memory_gb": 17.13, "ce": 0.26740942674223334, "kl": 0.017874155681056436, "context": 0.27634651435073465}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.3379565179347992, "loss_scale": 4.0, "retries": 0, "seconds": 62.4, "max_memory_gb": 17.13, "ce": 0.010805230762343854, "kl": 0.03491929391748272, "context": 0.02826487773563713}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 1.304590106010437, "loss_scale": 4.0, "retries": 0, "seconds": 63.9, "max_memory_gb": 17.13, "ce": 0.08102475036866963, "kl": 0.003085909818764776, "context": 0.08256770856678486}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 1.7246720790863037, "loss_scale": 4.0, "retries": 0, "seconds": 65.4, "max_memory_gb": 17.13, "ce": 0.08833921203040518, "kl": 0.0606912979419576, "context": 0.11868486099410802}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 2.726973533630371, "loss_scale": 4.0, "retries": 0, "seconds": 67.1, "max_memory_gb": 17.13, "ce": 0.3726605176925659, "kl": 0.12621153821237385, "context": 0.43576630204916}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 0.3080887794494629, "loss_scale": 4.0, "retries": 0, "seconds": 68.7, "max_memory_gb": 17.13, "ce": 0.01370291828061454, "kl": 0.003849382614134811, "context": 0.015627609798684716}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 1.5064500570297241, "loss_scale": 4.0, "retries": 0, "seconds": 70.1, "max_memory_gb": 17.13, "ce": 0.04806836260831915, "kl": 0.496382188051939, "context": 0.29625945165753365}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 2.1626319885253906, "loss_scale": 4.0, "retries": 0, "seconds": 71.6, "max_memory_gb": 17.13, "ce": 0.009769481184775941, "kl": 0.6864200670825085, "context": 0.3529795128852129}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.08964094519615173, "loss_scale": 4.0, "retries": 0, "seconds": 73.0, "max_memory_gb": 17.13, "ce": 0.006036847014911473, "kl": 0.0014478702360065654, "context": 0.006760782212950289}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.181659996509552, "loss_scale": 4.0, "retries": 0, "seconds": 74.5, "max_memory_gb": 17.13, "ce": 0.01442882648552768, "kl": 0.00683223577652825, "context": 0.017844944610260427}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 1.6251884698867798, "loss_scale": 4.0, "retries": 0, "seconds": 76.0, "max_memory_gb": 17.13, "ce": 0.06454130029305816, "kl": 0.03446829204040114, "context": 0.08177544892532751}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.1054396852850914, "loss_scale": 4.0, "retries": 0, "seconds": 77.5, "max_memory_gb": 17.13, "ce": 0.004281810077372938, "kl": 0.004943808809912298, "context": 0.006753714464139193}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 1.906571865081787, "loss_scale": 4.0, "retries": 0, "seconds": 78.9, "max_memory_gb": 17.13, "ce": 0.059482951008249074, "kl": 0.07820395057933638, "context": 0.0985849296557717}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 3.626570701599121, "loss_scale": 4.0, "retries": 0, "seconds": 80.3, "max_memory_gb": 17.13, "ce": 0.40603641944471747, "kl": 0.32497368403710425, "context": 0.5685232314281166}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 3.2724781036376953, "loss_scale": 4.0, "retries": 0, "seconds": 81.8, "max_memory_gb": 17.13, "ce": 0.46630698260560166, "kl": 0.4172554760007188, "context": 0.6749347083823523}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 2.499729633331299, "loss_scale": 4.0, "retries": 0, "seconds": 83.2, "max_memory_gb": 17.13, "ce": 0.2793736587045714, "kl": 0.18253209663089365, "context": 0.37063970672897995}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 7.299413681030273, "loss_scale": 4.0, "retries": 0, "seconds": 84.7, "max_memory_gb": 17.13, "ce": 0.2801815813872963, "kl": 0.059362948246416636, "context": 0.30986306245904416}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.8398693203926086, "loss_scale": 4.0, "retries": 0, "seconds": 86.1, "max_memory_gb": 17.13, "ce": 0.0054567999904975295, "kl": 0.11430235460284166, "context": 0.06260797585127875}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 1.5948010683059692, "loss_scale": 4.0, "retries": 0, "seconds": 87.7, "max_memory_gb": 17.13, "ce": 0.018131243406969588, "kl": 0.5063238861621358, "context": 0.2712931854184717}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.6280282735824585, "loss_scale": 4.0, "retries": 0, "seconds": 89.1, "max_memory_gb": 17.13, "ce": 0.003928378333512228, "kl": 0.12440971153409919, "context": 0.06613323272904381}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.45265746116638184, "loss_scale": 4.0, "retries": 0, "seconds": 90.6, "max_memory_gb": 17.13, "ce": 0.01832254562759772, "kl": 0.010130824382940773, "context": 0.0233879582665395}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 0.4709046483039856, "loss_scale": 4.0, "retries": 0, "seconds": 92.0, "max_memory_gb": 17.13, "ce": 0.024500716070178896, "kl": 0.02374572432017885, "context": 0.03637357917614281}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 3.8547403812408447, "loss_scale": 4.0, "retries": 0, "seconds": 93.5, "max_memory_gb": 17.13, "ce": 0.1696368174161762, "kl": 0.15606390210450627, "context": 0.2476687713060528}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 2.48856782913208, "loss_scale": 4.0, "retries": 0, "seconds": 94.9, "max_memory_gb": 17.13, "ce": 0.3217037992144469, "kl": 0.03438663621636806, "context": 0.3388971229142044}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 0.36347121000289917, "loss_scale": 4.0, "retries": 0, "seconds": 96.4, "max_memory_gb": 17.13, "ce": 0.0419672440548311, "kl": 0.01366928128481959, "context": 0.04880188376046135}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 1.7996001243591309, "loss_scale": 4.0, "retries": 0, "seconds": 97.9, "max_memory_gb": 17.13, "ce": 0.2057814250511001, "kl": 0.04962047829758376, "context": 0.2305916608311236}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 1.137416124343872, "loss_scale": 4.0, "retries": 0, "seconds": 99.4, "max_memory_gb": 17.13, "ce": 0.05201853404287249, "kl": 0.009351769782369956, "context": 0.05669441830832511}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 0.2828900218009949, "loss_scale": 4.0, "retries": 0, "seconds": 100.8, "max_memory_gb": 17.13, "ce": 0.009465347015066072, "kl": 0.004039407387608662, "context": 0.011485051189083606}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 1.6809697151184082, "loss_scale": 4.0, "retries": 0, "seconds": 102.3, "max_memory_gb": 17.13, "ce": 0.26404697087127715, "kl": 0.02821567526552826, "context": 0.27815480693243444}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.6637477874755859, "loss_scale": 4.0, "retries": 0, "seconds": 103.7, "max_memory_gb": 17.13, "ce": 0.038550138619029894, "kl": 0.010593462793622166, "context": 0.043846869957633317}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 0.8152750134468079, "loss_scale": 4.0, "retries": 0, "seconds": 105.3, "max_memory_gb": 17.13, "ce": 0.06433680687769083, "kl": 0.10334780901757767, "context": 0.11601071513723582}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.4213279187679291, "loss_scale": 4.0, "retries": 0, "seconds": 106.7, "max_memory_gb": 17.13, "ce": 0.013050014182226732, "kl": 0.05147689329805871, "context": 0.038788460806244984}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.49943655729293823, "loss_scale": 4.0, "retries": 0, "seconds": 108.2, "max_memory_gb": 17.13, "ce": 0.003045731515157968, "kl": 0.05425536964958155, "context": 0.030173415725585073}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.2654440104961395, "loss_scale": 4.0, "retries": 0, "seconds": 109.6, "max_memory_gb": 17.13, "ce": 0.011937780916923657, "kl": 0.014182532999257091, "context": 0.019029046583455056}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 2.1446778774261475, "loss_scale": 4.0, "retries": 0, "seconds": 111.2, "max_memory_gb": 17.13, "ce": 0.029143076622858644, "kl": 0.2380545488340431, "context": 0.14817035288433544}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.8618234395980835, "loss_scale": 4.0, "retries": 0, "seconds": 112.6, "max_memory_gb": 17.13, "ce": 0.06402293767314404, "kl": 0.030095580230408814, "context": 0.07907072582747787}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 0.7810187935829163, "loss_scale": 4.0, "retries": 0, "seconds": 114.1, "max_memory_gb": 17.13, "ce": 0.010334878537832992, "kl": 0.06784964915641467, "context": 0.04425970261218026}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 1.2665050029754639, "loss_scale": 4.0, "retries": 0, "seconds": 115.6, "max_memory_gb": 17.13, "ce": 0.3847197211580351, "kl": 0.05227493190500354, "context": 0.41085719081456773}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.36152493953704834, "loss_scale": 4.0, "retries": 0, "seconds": 117.1, "max_memory_gb": 17.13, "ce": 0.001565674290759489, "kl": 0.04041789856273681, "context": 0.021774623019155115}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 1.008164882659912, "loss_scale": 4.0, "retries": 0, "seconds": 118.5, "max_memory_gb": 17.13, "ce": 0.0044698363926727325, "kl": 0.6874795077856106, "context": 0.34820957767078653}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.15684716403484344, "loss_scale": 4.0, "retries": 0, "seconds": 120.1, "max_memory_gb": 17.13, "ce": 0.011164099909365177, "kl": 0.004314470163080841, "context": 0.01332133449614048}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 1.762632131576538, "loss_scale": 4.0, "retries": 0, "seconds": 121.5, "max_memory_gb": 17.13, "ce": 0.21621893588235253, "kl": 0.039103344457544154, "context": 0.2357706156108179}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 1.8594274520874023, "loss_scale": 4.0, "retries": 0, "seconds": 123.0, "max_memory_gb": 17.13, "ce": 0.17676432971347822, "kl": 0.042207288368445006, "context": 0.19786797581764404}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.3064635396003723, "loss_scale": 4.0, "retries": 0, "seconds": 124.4, "max_memory_gb": 17.13, "ce": 0.01651345263235271, "kl": 0.008036377254029503, "context": 0.02053164120297879}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.212511658668518, "loss_scale": 4.0, "retries": 0, "seconds": 125.8, "max_memory_gb": 17.13, "ce": 0.0017044533815351315, "kl": 0.1505001319208077, "context": 0.07695451788094942}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 1.5838403701782227, "loss_scale": 4.0, "retries": 0, "seconds": 127.3, "max_memory_gb": 17.13, "ce": 0.00397700946268742, "kl": 0.28642087086791435, "context": 0.14718744926722138}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.356819212436676, "loss_scale": 4.0, "retries": 0, "seconds": 128.9, "max_memory_gb": 17.13, "ce": 0.010536803267314099, "kl": 0.010931953747331136, "context": 0.01600278057230753}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.6593478322029114, "loss_scale": 4.0, "retries": 0, "seconds": 130.3, "max_memory_gb": 17.13, "ce": 0.006494704780379834, "kl": 0.40276773718142067, "context": 0.20787857105460716}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.1290024220943451, "loss_scale": 4.0, "retries": 0, "seconds": 131.9, "max_memory_gb": 17.13, "ce": 0.006171464476210531, "kl": 0.006875998848045128, "context": 0.00960946344275726}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.6968526840209961, "loss_scale": 4.0, "retries": 0, "seconds": 133.3, "max_memory_gb": 17.13, "ce": 0.028546748759254115, "kl": 0.04211310059463358, "context": 0.0496032990704407}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 1.5876822471618652, "loss_scale": 4.0, "retries": 0, "seconds": 134.7, "max_memory_gb": 17.13, "ce": 0.09763055479561444, "kl": 0.1093532175391374, "context": 0.15230715984944254}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.5713454484939575, "loss_scale": 4.0, "retries": 0, "seconds": 136.2, "max_memory_gb": 17.13, "ce": 0.050021154806017876, "kl": 0.05963436969614122, "context": 0.07983833967591636}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 2.304703950881958, "loss_scale": 4.0, "retries": 0, "seconds": 137.6, "max_memory_gb": 17.13, "ce": 0.013718259986490011, "kl": 0.18343693545466522, "context": 0.1054367262404412}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 0.7144511938095093, "loss_scale": 4.0, "retries": 0, "seconds": 139.1, "max_memory_gb": 17.13, "ce": 0.04000171933148522, "kl": 0.11547503722613328, "context": 0.09773923981992994}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 1.8347748517990112, "loss_scale": 4.0, "retries": 0, "seconds": 140.6, "max_memory_gb": 17.13, "ce": 0.17706307425396517, "kl": 0.30676539428532124, "context": 0.33044576714746654}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 0.06928425282239914, "loss_scale": 4.0, "retries": 0, "seconds": 142.1, "max_memory_gb": 17.13, "ce": 0.004257451029843651, "kl": 0.0014615143245464424, "context": 0.00498820815118961}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 2.9674952030181885, "loss_scale": 4.0, "retries": 0, "seconds": 143.5, "max_memory_gb": 17.13, "ce": 0.1871682815872191, "kl": 0.750408578342558, "context": 0.5623725671321154}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 0.4398840367794037, "loss_scale": 4.0, "retries": 0, "seconds": 145.1, "max_memory_gb": 17.13, "ce": 0.02071583232282137, "kl": 0.04734901600750163, "context": 0.0443903396371752}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.3389618992805481, "loss_scale": 4.0, "retries": 0, "seconds": 146.5, "max_memory_gb": 17.13, "ce": 0.015076578827574849, "kl": 0.02121697464463068, "context": 0.025685066182632}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.2078893631696701, "loss_scale": 4.0, "retries": 0, "seconds": 148.1, "max_memory_gb": 17.13, "ce": 0.012975801946595311, "kl": 0.010747079733846476, "context": 0.018349341466091573}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 1.346603512763977, "loss_scale": 4.0, "retries": 0, "seconds": 149.5, "max_memory_gb": 17.13, "ce": 0.07807047010282986, "kl": 0.07119606976448267, "context": 0.11366850871127099}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.7031606435775757, "loss_scale": 4.0, "retries": 0, "seconds": 151.0, "max_memory_gb": 17.13, "ce": 0.05239143769722432, "kl": 0.06116955683683045, "context": 0.08297621988458559}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 1.9982140064239502, "loss_scale": 4.0, "retries": 0, "seconds": 152.4, "max_memory_gb": 17.13, "ce": 0.2539606852806173, "kl": 0.054733975943236146, "context": 0.28132768138311803}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 1.6031965017318726, "loss_scale": 4.0, "retries": 0, "seconds": 154.0, "max_memory_gb": 17.13, "ce": 0.07768587733153254, "kl": 0.025016853673150763, "context": 0.09019430266926065}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.07153131812810898, "loss_scale": 4.0, "retries": 0, "seconds": 155.4, "max_memory_gb": 17.13, "ce": 0.005916070949751884, "kl": 0.002338781676144208, "context": 0.007085462042596191}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 2.2280566692352295, "loss_scale": 4.0, "retries": 0, "seconds": 156.9, "max_memory_gb": 17.13, "ce": 0.20151554546464467, "kl": 0.04027486575068906, "context": 0.22165298159234226}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 1.536313533782959, "loss_scale": 4.0, "retries": 0, "seconds": 158.3, "max_memory_gb": 17.13, "ce": 0.15819375892169774, "kl": 0.05254438941483386, "context": 0.18446595640853047}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.917443573474884, "loss_scale": 4.0, "retries": 0, "seconds": 159.9, "max_memory_gb": 17.13, "ce": 0.01866378163686022, "kl": 0.15101159512414597, "context": 0.09416957816574723}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.1586960405111313, "loss_scale": 4.0, "retries": 0, "seconds": 161.3, "max_memory_gb": 17.13, "ce": 0.0020260598212189507, "kl": 0.012125978746553301, "context": 0.008089049020782113}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 0.17599162459373474, "loss_scale": 4.0, "retries": 0, "seconds": 162.8, "max_memory_gb": 17.13, "ce": 0.013860665960237384, "kl": 0.005513100069947541, "context": 0.016617216053418815}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.08533327281475067, "loss_scale": 4.0, "retries": 0, "seconds": 164.2, "max_memory_gb": 17.13, "ce": 0.0023969212124939077, "kl": 0.007694279425777495, "context": 0.006244061136385426}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.24502050876617432, "loss_scale": 4.0, "retries": 0, "seconds": 165.7, "max_memory_gb": 17.13, "ce": 0.007145042560296133, "kl": 0.00315972564931144, "context": 0.008724905404960737}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 2.001430034637451, "loss_scale": 4.0, "retries": 0, "seconds": 167.1, "max_memory_gb": 17.13, "ce": 0.1264552955617546, "kl": 0.0009210805101247388, "context": 0.1269158291979693}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.45022135972976685, "loss_scale": 4.0, "retries": 0, "seconds": 168.5, "max_memory_gb": 17.13, "ce": 0.02339838091575075, "kl": 0.02292733838658023, "context": 0.034862050975789316}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 2.604278564453125, "loss_scale": 4.0, "retries": 0, "seconds": 170.1, "max_memory_gb": 17.13, "ce": 0.6276867958076764, "kl": 0.031767930769092345, "context": 0.6435707650089171}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 1.2729394435882568, "loss_scale": 4.0, "retries": 0, "seconds": 171.5, "max_memory_gb": 17.13, "ce": 0.20215764780004974, "kl": 0.10464468924328685, "context": 0.2544799887109548}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 0.7664515376091003, "loss_scale": 4.0, "retries": 0, "seconds": 173.0, "max_memory_gb": 17.13, "ce": 0.037545231432886794, "kl": 0.0158048842736207, "context": 0.04544767356128432}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 3.162247896194458, "loss_scale": 4.0, "retries": 0, "seconds": 174.4, "max_memory_gb": 17.13, "ce": 0.4006639693980105, "kl": 0.018479381884390023, "context": 0.40990367386257276}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.5416079759597778, "loss_scale": 4.0, "retries": 0, "seconds": 175.9, "max_memory_gb": 17.13, "ce": 0.009245573353837244, "kl": 0.013027107230300317, "context": 0.015759127440105658}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 0.7410915493965149, "loss_scale": 4.0, "retries": 0, "seconds": 177.3, "max_memory_gb": 17.13, "ce": 0.02247010450810194, "kl": 0.019411623521591537, "context": 0.03217591543216258}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.2234264761209488, "loss_scale": 4.0, "retries": 0, "seconds": 178.9, "max_memory_gb": 17.13, "ce": 0.00806631516024936, "kl": 0.0006096675315347966, "context": 0.00837114879686851}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 1.6292756795883179, "loss_scale": 4.0, "retries": 0, "seconds": 180.3, "max_memory_gb": 17.13, "ce": 0.1751544252038002, "kl": 0.13780904316809028, "context": 0.24405894218944013}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 1.316921591758728, "loss_scale": 4.0, "retries": 0, "seconds": 181.8, "max_memory_gb": 17.13, "ce": 0.018616623914567754, "kl": 0.6581434306135634, "context": 0.34768833182170056}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 2.247457504272461, "loss_scale": 4.0, "retries": 0, "seconds": 183.2, "max_memory_gb": 17.13, "ce": 0.08764443903055508, "kl": 0.013513580086510046, "context": 0.09440122806699947}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 4.43902587890625, "loss_scale": 4.0, "retries": 0, "seconds": 184.7, "max_memory_gb": 17.13, "ce": 0.39408774723415263, "kl": 0.06306334088185395, "context": 0.42561941401072545}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 0.11447470635175705, "loss_scale": 4.0, "retries": 0, "seconds": 186.2, "max_memory_gb": 17.13, "ce": 0.0009167608441202901, "kl": 0.013623641672893427, "context": 0.00772858178243041}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 2.01371169090271, "loss_scale": 4.0, "retries": 0, "seconds": 187.7, "max_memory_gb": 17.13, "ce": 0.08738659182563424, "kl": 1.201492253690958, "context": 0.6881327144801617}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 0.5986687541007996, "loss_scale": 4.0, "retries": 0, "seconds": 189.1, "max_memory_gb": 17.13, "ce": 0.023293798905797303, "kl": 0.028329626191407442, "context": 0.03745861240895465}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.025192538276314735, "loss_scale": 4.0, "retries": 0, "seconds": 190.6, "max_memory_gb": 17.13, "ce": 0.0013391276224865578, "kl": 0.0015038976580399321, "context": 0.002091076457872987}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.1876341551542282, "loss_scale": 4.0, "retries": 0, "seconds": 192.0, "max_memory_gb": 17.13, "ce": 0.008225322002544999, "kl": 0.022095244494266808, "context": 0.019272944307886064}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 1.1161683797836304, "loss_scale": 4.0, "retries": 0, "seconds": 193.6, "max_memory_gb": 17.13, "ce": 0.053675013390602544, "kl": 0.017998073373746593, "context": 0.06267405101971235}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 1.7366355657577515, "loss_scale": 4.0, "retries": 0, "seconds": 195.0, "max_memory_gb": 17.13, "ce": 0.07200070295948535, "kl": 0.10158892848994583, "context": 0.12279516528360546}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 1.1162179708480835, "loss_scale": 4.0, "retries": 0, "seconds": 196.5, "max_memory_gb": 17.13, "ce": 0.05655742299859412, "kl": 0.24963634246159927, "context": 0.18137559424212668}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.597100019454956, "loss_scale": 4.0, "retries": 0, "seconds": 197.9, "max_memory_gb": 17.13, "ce": 0.021356292359996587, "kl": 0.017438674003642518, "context": 0.030075630405917764}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.5955108404159546, "loss_scale": 4.0, "retries": 0, "seconds": 199.4, "max_memory_gb": 17.13, "ce": 0.04440073606383521, "kl": 0.0638392202608884, "context": 0.07632034999551252}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.10679948329925537, "loss_scale": 4.0, "retries": 0, "seconds": 200.8, "max_memory_gb": 17.13, "ce": 0.004899157502222806, "kl": 0.0017145962342510757, "context": 0.005756455648224801}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 2.792469024658203, "loss_scale": 4.0, "retries": 0, "seconds": 202.4, "max_memory_gb": 17.13, "ce": 0.09890787527547218, "kl": 0.000991866842014133, "context": 0.09940380696207285}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 6.513494491577148, "loss_scale": 4.0, "retries": 0, "seconds": 203.8, "max_memory_gb": 17.13, "ce": 0.7893244657898322, "kl": 0.26732675574021414, "context": 0.9229878361802548}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 4.048924922943115, "loss_scale": 4.0, "retries": 0, "seconds": 205.3, "max_memory_gb": 17.13, "ce": 0.13305957743432373, "kl": 0.002746905220192275, "context": 0.13443302491214126}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.18106578290462494, "loss_scale": 4.0, "retries": 0, "seconds": 206.7, "max_memory_gb": 17.13, "ce": 0.008959031227277592, "kl": 0.004662235245632473, "context": 0.011290149093838409}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 0.08351907134056091, "loss_scale": 4.0, "retries": 0, "seconds": 208.3, "max_memory_gb": 17.13, "ce": 0.004699777811765671, "kl": 0.007089560298481956, "context": 0.008244557771831751}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.5485881567001343, "loss_scale": 4.0, "retries": 0, "seconds": 209.7, "max_memory_gb": 17.13, "ce": 0.0466169175633695, "kl": 0.0038389458568417467, "context": 0.04853639187058434}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.07088983058929443, "loss_scale": 4.0, "retries": 0, "seconds": 211.2, "max_memory_gb": 17.13, "ce": 0.004742231001728214, "kl": 0.007227590596812661, "context": 0.008356026039109565}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 1.111382007598877, "loss_scale": 4.0, "retries": 0, "seconds": 212.7, "max_memory_gb": 17.13, "ce": 0.1634286453772802, "kl": 0.06012822187040001, "context": 0.19349276169668883}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 2.1561126708984375, "loss_scale": 4.0, "retries": 0, "seconds": 214.1, "max_memory_gb": 17.13, "ce": 0.10560522705782205, "kl": 0.3913388365724586, "context": 0.301274652214488}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.20143961906433105, "loss_scale": 4.0, "retries": 0, "seconds": 215.6, "max_memory_gb": 17.13, "ce": 0.007085574645316228, "kl": 0.02382507122001698, "context": 0.018998110463144258}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 1.9381755590438843, "loss_scale": 4.0, "retries": 0, "seconds": 217.2, "max_memory_gb": 17.13, "ce": 0.13673302141251042, "kl": 0.24190336500760168, "context": 0.25768470391631126}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 2.3145499229431152, "loss_scale": 4.0, "retries": 0, "seconds": 218.6, "max_memory_gb": 17.13, "ce": 0.28246601368300617, "kl": 0.013866306922864169, "context": 0.28939916542731225}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 2.735178232192993, "loss_scale": 4.0, "retries": 0, "seconds": 220.1, "max_memory_gb": 17.13, "ce": 0.1642060384619981, "kl": 0.00024054500318015926, "context": 0.16432631463976577}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.38414737582206726, "loss_scale": 4.0, "retries": 0, "seconds": 221.5, "max_memory_gb": 17.13, "ce": 0.0010899825902015436, "kl": 0.03921228128456278, "context": 0.020696122344816104}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 3.304277181625366, "loss_scale": 4.0, "retries": 0, "seconds": 222.9, "max_memory_gb": 17.13, "ce": 0.8666802205261774, "kl": 0.05437706588418223, "context": 0.8938687504269183}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 2.0954625606536865, "loss_scale": 4.0, "retries": 0, "seconds": 224.5, "max_memory_gb": 17.13, "ce": 0.007556533761089668, "kl": 0.3625934578012675, "context": 0.1888532629236579}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 1.0164474248886108, "loss_scale": 4.0, "retries": 0, "seconds": 225.8, "max_memory_gb": 17.13, "ce": 0.1786849199270364, "kl": 0.0564800394440681, "context": 0.20692493382375687}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 1.9871703386306763, "loss_scale": 4.0, "retries": 0, "seconds": 227.4, "max_memory_gb": 17.13, "ce": 0.08857645166790462, "kl": 0.6567266639322042, "context": 0.41693978069815785}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.047279756516218185, "loss_scale": 4.0, "retries": 0, "seconds": 228.8, "max_memory_gb": 17.13, "ce": 0.002850180637324229, "kl": 0.0022484350856757374, "context": 0.003974398219725117}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 1.1509376764297485, "loss_scale": 4.0, "retries": 0, "seconds": 230.4, "max_memory_gb": 17.13, "ce": 0.06318757072585868, "kl": 0.030312597115880635, "context": 0.0783438684848079}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.3467220366001129, "loss_scale": 4.0, "retries": 0, "seconds": 231.7, "max_memory_gb": 17.13, "ce": 0.014116479476797394, "kl": 0.016552289092487626, "context": 0.022392624989151955}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.3325499892234802, "loss_scale": 4.0, "retries": 0, "seconds": 233.3, "max_memory_gb": 17.13, "ce": 0.025531076651532203, "kl": 0.010873172723222524, "context": 0.030967663042247295}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 1.767264723777771, "loss_scale": 4.0, "retries": 0, "seconds": 234.7, "max_memory_gb": 17.13, "ce": 0.2635727181332186, "kl": 0.29545822447835235, "context": 0.4113018229836598}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 0.11883631348609924, "loss_scale": 4.0, "retries": 0, "seconds": 236.3, "max_memory_gb": 17.13, "ce": 0.0043409507161413785, "kl": 0.010929162894171895, "context": 0.009805531917663757}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.023011181503534317, "loss_scale": 4.0, "retries": 0, "seconds": 237.7, "max_memory_gb": 17.13, "ce": 0.0015794880164321512, "kl": 0.00037022414107923396, "context": 0.0017646000924287364}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.9211810231208801, "loss_scale": 4.0, "retries": 0, "seconds": 239.2, "max_memory_gb": 17.13, "ce": 0.005328447849024087, "kl": 0.3250726313199266, "context": 0.16786477035202552}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 0.8289332985877991, "loss_scale": 4.0, "retries": 0, "seconds": 240.6, "max_memory_gb": 17.13, "ce": 0.06359092608909123, "kl": 0.11182437982824922, "context": 0.11950311379041523}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 0.55936199426651, "loss_scale": 4.0, "retries": 0, "seconds": 242.2, "max_memory_gb": 17.13, "ce": 0.018386251060292125, "kl": 0.06040382961509749, "context": 0.04858816578052938}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 0.8010367155075073, "loss_scale": 4.0, "retries": 0, "seconds": 243.6, "max_memory_gb": 17.13, "ce": 0.0278459359833505, "kl": 0.06185095587716205, "context": 0.058771413925569504}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.4553421437740326, "loss_scale": 4.0, "retries": 0, "seconds": 245.1, "max_memory_gb": 17.13, "ce": 0.03014357329811901, "kl": 0.020070215236955846, "context": 0.04017867991933599}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.05754156783223152, "loss_scale": 4.0, "retries": 0, "seconds": 246.5, "max_memory_gb": 17.13, "ce": 0.0030723828967893496, "kl": 0.0018904104435932823, "context": 0.004017588071292266}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.06749233603477478, "loss_scale": 4.0, "retries": 0, "seconds": 248.1, "max_memory_gb": 17.13, "ce": 0.005137685802765191, "kl": 0.0005177808125154115, "context": 0.005396576103521511}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 1.4259148836135864, "loss_scale": 4.0, "retries": 0, "seconds": 249.5, "max_memory_gb": 17.13, "ce": 0.34873294121643994, "kl": 0.012449261974325054, "context": 0.354957584218937}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 1.5646181106567383, "loss_scale": 4.0, "retries": 0, "seconds": 251.0, "max_memory_gb": 17.13, "ce": 0.17184380412800238, "kl": 0.11228767316788435, "context": 0.22798764053732157}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 0.09278389066457748, "loss_scale": 4.0, "retries": 0, "seconds": 252.4, "max_memory_gb": 17.13, "ce": 0.0051806195406243205, "kl": 0.0009382526150147896, "context": 0.005649745784467086}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 1.176147222518921, "loss_scale": 4.0, "retries": 0, "seconds": 254.0, "max_memory_gb": 17.13, "ce": 0.11331757786683738, "kl": 0.001682452184468275, "context": 0.11415880243293941}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 1.7309280633926392, "loss_scale": 4.0, "retries": 0, "seconds": 255.4, "max_memory_gb": 17.13, "ce": 0.08555068704299629, "kl": 0.11909660062883631, "context": 0.14509898738469929}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 2.4001235961914062, "loss_scale": 4.0, "retries": 0, "seconds": 256.8, "max_memory_gb": 17.13, "ce": 0.23269203281961381, "kl": 0.053580778767354786, "context": 0.2594824214465916}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.28959178924560547, "loss_scale": 4.0, "retries": 0, "seconds": 258.3, "max_memory_gb": 17.13, "ce": 0.04001123990747146, "kl": 0.013110229577222299, "context": 0.046566353674279526}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 2.1745429039001465, "loss_scale": 4.0, "retries": 0, "seconds": 259.7, "max_memory_gb": 17.13, "ce": 0.16075166337031987, "kl": 0.02698784963433809, "context": 0.1742455864114163}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 1.351664423942566, "loss_scale": 4.0, "retries": 0, "seconds": 261.3, "max_memory_gb": 17.13, "ce": 0.1490348633378744, "kl": 0.3616792691173032, "context": 0.3298745034262538}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 1.1799719333648682, "loss_scale": 4.0, "retries": 0, "seconds": 262.7, "max_memory_gb": 17.13, "ce": 0.14847401122096926, "kl": 0.03365168164600618, "context": 0.1652998492354527}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.8154199123382568, "loss_scale": 4.0, "retries": 0, "seconds": 264.2, "max_memory_gb": 17.13, "ce": 0.08636467478936538, "kl": 0.058442574775654066, "context": 0.11558595777023584}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.1428370475769043, "loss_scale": 4.0, "retries": 0, "seconds": 265.7, "max_memory_gb": 17.13, "ce": 0.008588679895183304, "kl": 0.0020714758408075795, "context": 0.00962441799492808}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 3.3667545318603516, "loss_scale": 4.0, "retries": 0, "seconds": 267.2, "max_memory_gb": 17.13, "ce": 0.07730326554155909, "kl": 0.017124197554949205, "context": 0.08586536350776441}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.4244900643825531, "loss_scale": 4.0, "retries": 0, "seconds": 268.6, "max_memory_gb": 17.13, "ce": 0.006683874511509202, "kl": 0.016789536382930237, "context": 0.01507864284212701}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 1.3712491989135742, "loss_scale": 4.0, "retries": 0, "seconds": 270.1, "max_memory_gb": 17.13, "ce": 0.1377375138399657, "kl": 0.02819188532885164, "context": 0.15183345810510218}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.875066876411438, "loss_scale": 4.0, "retries": 0, "seconds": 271.5, "max_memory_gb": 17.13, "ce": 0.002351022662878677, "kl": 0.09420104064702173, "context": 0.04945154128654394}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.7683429718017578, "loss_scale": 4.0, "retries": 0, "seconds": 273.1, "max_memory_gb": 17.13, "ce": 0.0038478271453641355, "kl": 0.1630499902767042, "context": 0.08537282072938979}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 1.534468412399292, "loss_scale": 4.0, "retries": 0, "seconds": 274.5, "max_memory_gb": 17.13, "ce": 0.22673552605556324, "kl": 0.046481570694595575, "context": 0.2499763104133308}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 1.4931929111480713, "loss_scale": 4.0, "retries": 0, "seconds": 276.0, "max_memory_gb": 17.13, "ce": 0.05585032507224241, "kl": 0.01081976901514281, "context": 0.06126020981173497}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 1.6403292417526245, "loss_scale": 4.0, "retries": 0, "seconds": 277.4, "max_memory_gb": 17.13, "ce": 0.012393177836202085, "kl": 0.1569221470854245, "context": 0.09085425251396373}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 1.4212841987609863, "loss_scale": 4.0, "retries": 0, "seconds": 279.0, "max_memory_gb": 17.13, "ce": 0.1026644459052477, "kl": 0.19341437063849298, "context": 0.19937163122813217}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 2.023947238922119, "loss_scale": 4.0, "retries": 0, "seconds": 280.4, "max_memory_gb": 17.13, "ce": 0.059952243900625035, "kl": 0.08636950239815633, "context": 0.10313699484686367}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 0.6165140271186829, "loss_scale": 4.0, "retries": 0, "seconds": 281.9, "max_memory_gb": 17.13, "ce": 0.004545347379462328, "kl": 0.12033358846201736, "context": 0.06471214245539159}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 6.917763710021973, "loss_scale": 4.0, "retries": 0, "seconds": 283.3, "max_memory_gb": 17.13, "ce": 0.1666569097833417, "kl": 0.1671803037988866, "context": 0.25024706182739465}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.04522839933633804, "loss_scale": 4.0, "retries": 0, "seconds": 284.8, "max_memory_gb": 17.13, "ce": 0.0035193708317819983, "kl": 0.0009185090348182712, "context": 0.003978625201852992}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 0.8387200832366943, "loss_scale": 4.0, "retries": 0, "seconds": 286.3, "max_memory_gb": 17.13, "ce": 0.008168972592102364, "kl": 0.07029617579974001, "context": 0.04331706021912396}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.2842733860015869, "loss_scale": 4.0, "retries": 0, "seconds": 287.8, "max_memory_gb": 17.13, "ce": 0.007428420343785547, "kl": 0.02012812875182135, "context": 0.01749248427222483}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 1.404988169670105, "loss_scale": 4.0, "retries": 0, "seconds": 289.2, "max_memory_gb": 17.13, "ce": 0.04304501433216501, "kl": 0.20054942208844295, "context": 0.143319723487366}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.9662405848503113, "loss_scale": 4.0, "retries": 0, "seconds": 290.8, "max_memory_gb": 17.13, "ce": 0.024006576652027434, "kl": 0.1772840586490929, "context": 0.11264860404480714}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.03017435409128666, "loss_scale": 4.0, "retries": 0, "seconds": 292.2, "max_memory_gb": 17.13, "ce": 0.0018214766751043499, "kl": 0.0015581039947392128, "context": 0.0026005286199506372}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.009941186755895615, "loss_scale": 4.0, "retries": 0, "seconds": 293.7, "max_memory_gb": 17.13, "ce": 0.0004908747650915757, "kl": 0.00015511897117903573, "context": 0.0005684342468157411}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 1.2086546421051025, "loss_scale": 4.0, "retries": 0, "seconds": 295.2, "max_memory_gb": 17.13, "ce": 0.05842914557433687, "kl": 0.3292131372872973, "context": 0.22303571790689602}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 0.04941399022936821, "loss_scale": 4.0, "retries": 0, "seconds": 296.7, "max_memory_gb": 17.13, "ce": 0.0016000101604731753, "kl": 0.0002033097102867032, "context": 0.001701664998108754}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.1087390184402466, "loss_scale": 4.0, "retries": 0, "seconds": 298.1, "max_memory_gb": 17.13, "ce": 0.014819476404227316, "kl": 0.7985869613476098, "context": 0.41411296278238297}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.44446682929992676, "loss_scale": 4.0, "retries": 0, "seconds": 299.6, "max_memory_gb": 17.13, "ce": 0.030375603862921707, "kl": 0.011529122879437637, "context": 0.03614016556821298}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.1973123848438263, "loss_scale": 4.0, "retries": 0, "seconds": 301.0, "max_memory_gb": 17.13, "ce": 0.02048086364447954, "kl": 0.0030864940895298787, "context": 0.022024110094207572}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 5.328816890716553, "loss_scale": 4.0, "retries": 0, "seconds": 302.4, "max_memory_gb": 17.13, "ce": 0.09469023073324934, "kl": 0.047821447951719165, "context": 0.11860095313750207}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.032633017748594284, "loss_scale": 4.0, "retries": 0, "seconds": 304.0, "max_memory_gb": 17.13, "ce": 0.002348562018596567, "kl": 0.00032719169212214183, "context": 0.002512157880119048}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 2.3913931846618652, "loss_scale": 4.0, "retries": 0, "seconds": 305.5, "max_memory_gb": 17.13, "ce": 0.33536645552521804, "kl": 0.007245238948598853, "context": 0.33898906801550766}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 1.8881717920303345, "loss_scale": 4.0, "retries": 0, "seconds": 306.9, "max_memory_gb": 17.13, "ce": 0.23459577592439018, "kl": 0.5086524897524214, "context": 0.4889220219629351}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.26593339443206787, "loss_scale": 4.0, "retries": 0, "seconds": 308.5, "max_memory_gb": 17.13, "ce": 0.013906762014812557, "kl": 0.0033715785500127993, "context": 0.015592551175359404}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.13302548229694366, "loss_scale": 4.0, "retries": 0, "seconds": 309.8, "max_memory_gb": 17.13, "ce": 0.012300061818677932, "kl": 0.0015688872135797283, "context": 0.013084505219012499}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.05647924914956093, "loss_scale": 4.0, "retries": 0, "seconds": 311.3, "max_memory_gb": 17.13, "ce": 0.00399148598080501, "kl": 0.002548932701756712, "context": 0.005265952320769429}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 1.0288753509521484, "loss_scale": 4.0, "retries": 0, "seconds": 312.8, "max_memory_gb": 17.13, "ce": 0.022947184916120023, "kl": 0.003921704600543308, "context": 0.024908037237764802}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 3.0675973892211914, "loss_scale": 4.0, "retries": 0, "seconds": 314.2, "max_memory_gb": 17.13, "ce": 0.5918415513260697, "kl": 0.17379318625899032, "context": 0.6787381360772997}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 1.561137318611145, "loss_scale": 4.0, "retries": 0, "seconds": 315.9, "max_memory_gb": 17.13, "ce": 0.046069449113929295, "kl": 0.14403752970611094, "context": 0.11808821302838624}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 1.301905632019043, "loss_scale": 4.0, "retries": 0, "seconds": 317.3, "max_memory_gb": 17.13, "ce": 0.07360655110096559, "kl": 0.053843383386265486, "context": 0.10052823950536549}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.5765725374221802, "loss_scale": 4.0, "retries": 0, "seconds": 318.9, "max_memory_gb": 17.13, "ce": 0.05754157330375165, "kl": 0.037081423550262116, "context": 0.07608228153549135}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.19971433281898499, "loss_scale": 4.0, "retries": 0, "seconds": 320.4, "max_memory_gb": 17.13, "ce": 0.012768677974236198, "kl": 0.005060843581304653, "context": 0.015299099526600912}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.5891904234886169, "loss_scale": 4.0, "retries": 0, "seconds": 321.9, "max_memory_gb": 17.13, "ce": 0.007721015514107421, "kl": 0.11906722089577215, "context": 0.06725462572649121}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.3511645495891571, "loss_scale": 4.0, "retries": 0, "seconds": 323.3, "max_memory_gb": 17.13, "ce": 0.023765893100062385, "kl": 0.004040211788378656, "context": 0.02578599876142107}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 0.5770818591117859, "loss_scale": 4.0, "retries": 0, "seconds": 324.9, "max_memory_gb": 17.13, "ce": 0.09187292262868141, "kl": 0.03186998967839827, "context": 0.10780791744036833}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.8543387055397034, "loss_scale": 4.0, "retries": 0, "seconds": 326.3, "max_memory_gb": 17.13, "ce": 0.04273714529699646, "kl": 0.000536231258593034, "context": 0.043005262792576104}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 1.178795576095581, "loss_scale": 4.0, "retries": 0, "seconds": 327.8, "max_memory_gb": 17.13, "ce": 0.03391592692787526, "kl": 0.3391564459379879, "context": 0.20349415528471582}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.5855685472488403, "loss_scale": 4.0, "retries": 0, "seconds": 329.2, "max_memory_gb": 17.13, "ce": 0.027793962020950858, "kl": 0.009310575928338949, "context": 0.032449250749778}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.9845817685127258, "loss_scale": 4.0, "retries": 0, "seconds": 330.7, "max_memory_gb": 17.13, "ce": 0.08692189591238275, "kl": 0.06182130343040626, "context": 0.11783255339832976}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 1.1817957162857056, "loss_scale": 4.0, "retries": 0, "seconds": 332.1, "max_memory_gb": 17.13, "ce": 0.005438613996375352, "kl": 0.39870703244775996, "context": 0.20479212753707543}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 1.3052387237548828, "loss_scale": 4.0, "retries": 0, "seconds": 333.7, "max_memory_gb": 17.13, "ce": 0.03819104887224967, "kl": 0.07308128308102368, "context": 0.07473169412696734}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 1.0245038270950317, "loss_scale": 4.0, "retries": 0, "seconds": 335.1, "max_memory_gb": 17.13, "ce": 0.04489440740871942, "kl": 0.0901946898402457, "context": 0.08999175212375121}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.6381441354751587, "loss_scale": 4.0, "retries": 0, "seconds": 336.6, "max_memory_gb": 17.13, "ce": 0.04286300034436863, "kl": 0.01766058699149653, "context": 0.05169329361524433}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 1.394091248512268, "loss_scale": 4.0, "retries": 0, "seconds": 338.0, "max_memory_gb": 17.13, "ce": 0.011397933325497434, "kl": 0.15854429692262784, "context": 0.09067007945850492}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 2.1059317588806152, "loss_scale": 4.0, "retries": 0, "seconds": 339.6, "max_memory_gb": 17.13, "ce": 0.03488610323984176, "kl": 0.5171158026769263, "context": 0.29344400609261356}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 0.04043993726372719, "loss_scale": 4.0, "retries": 0, "seconds": 341.0, "max_memory_gb": 17.13, "ce": 0.002860502354451455, "kl": 0.0008305270912387641, "context": 0.0032757659209892154}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.31420525908470154, "loss_scale": 4.0, "retries": 0, "seconds": 342.6, "max_memory_gb": 17.13, "ce": 0.01123914332856657, "kl": 0.015408702272452501, "context": 0.018943494927952997}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 1.4317739009857178, "loss_scale": 4.0, "retries": 0, "seconds": 344.1, "max_memory_gb": 17.13, "ce": 0.08339154195527954, "kl": 0.170698172878474, "context": 0.168740626424551}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.025034507736563683, "loss_scale": 4.0, "retries": 0, "seconds": 345.6, "max_memory_gb": 17.13, "ce": 0.0015635209347237833, "kl": 0.00028438419803933357, "context": 0.0017057130753528327}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 1.3368520736694336, "loss_scale": 4.0, "retries": 0, "seconds": 347.2, "max_memory_gb": 17.13, "ce": 0.07675701804691926, "kl": 0.06795056876762828, "context": 0.11073230241890997}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 3.099306106567383, "loss_scale": 4.0, "retries": 0, "seconds": 348.7, "max_memory_gb": 17.13, "ce": 0.09238692550570704, "kl": 0.32697166420985013, "context": 0.2558727639261633}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 2.3414463996887207, "loss_scale": 4.0, "retries": 0, "seconds": 350.2, "max_memory_gb": 17.13, "ce": 0.8269484224292682, "kl": 0.011771404673709185, "context": 0.8328341210144572}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.16139884293079376, "loss_scale": 4.0, "retries": 0, "seconds": 351.6, "max_memory_gb": 17.13, "ce": 0.010664274799637496, "kl": 0.005403209004725795, "context": 0.01336587946570944}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.7421970367431641, "loss_scale": 4.0, "retries": 0, "seconds": 353.1, "max_memory_gb": 17.13, "ce": 0.020221955346642062, "kl": 0.0007577781798318028, "context": 0.020600843854481354}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.17356327176094055, "loss_scale": 4.0, "retries": 0, "seconds": 354.4, "max_memory_gb": 17.13, "ce": 0.001930893849930726, "kl": 0.01720941844951085, "context": 0.010535603330936283}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 1.2163132429122925, "loss_scale": 4.0, "retries": 0, "seconds": 355.8, "max_memory_gb": 17.13, "ce": 0.024800429528113455, "kl": 0.45227873788098805, "context": 0.25093980063684285}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 1.5966113805770874, "loss_scale": 4.0, "retries": 0, "seconds": 357.0, "max_memory_gb": 17.13, "ce": 0.04995183789287694, "kl": 0.05320354994273657, "context": 0.0765536091348622}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 1.0162805318832397, "loss_scale": 4.0, "retries": 0, "seconds": 358.5, "max_memory_gb": 17.13, "ce": 0.13641435427416582, "kl": 0.039752283146299305, "context": 0.1562904921156587}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.42014452815055847, "loss_scale": 4.0, "retries": 0, "seconds": 359.7, "max_memory_gb": 17.13, "ce": 0.047858222416834906, "kl": 0.014429960800043773, "context": 0.05507320334436372}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 1.3861477375030518, "loss_scale": 4.0, "retries": 0, "seconds": 361.3, "max_memory_gb": 17.13, "ce": 0.10853369998949347, "kl": 0.017316350582404993, "context": 0.11719187740527559}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 1.6047312021255493, "loss_scale": 4.0, "retries": 0, "seconds": 362.7, "max_memory_gb": 17.13, "ce": 0.055389323679264635, "kl": 0.03127224884519819, "context": 0.0710254482110031}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.506386399269104, "loss_scale": 4.0, "retries": 0, "seconds": 364.3, "max_memory_gb": 17.13, "ce": 0.03856670622189995, "kl": 0.007590379328576091, "context": 0.04236189472430851}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.9252721071243286, "loss_scale": 4.0, "retries": 0, "seconds": 365.8, "max_memory_gb": 17.13, "ce": 0.18316892461734824, "kl": 0.01216891227522865, "context": 0.18925337842665613}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 2.199199914932251, "loss_scale": 4.0, "retries": 0, "seconds": 367.3, "max_memory_gb": 17.13, "ce": 0.05522972164908424, "kl": 0.08535341461538337, "context": 0.09790642897132784}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.04854186624288559, "loss_scale": 4.0, "retries": 0, "seconds": 368.8, "max_memory_gb": 17.13, "ce": 0.0019207662626286037, "kl": 0.002011148241308547, "context": 0.0029263403557706624}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.3526778817176819, "loss_scale": 4.0, "retries": 0, "seconds": 370.3, "max_memory_gb": 17.13, "ce": 0.021150173008209094, "kl": 0.0197021758685878, "context": 0.03100126181379892}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 1.448015809059143, "loss_scale": 4.0, "retries": 0, "seconds": 371.8, "max_memory_gb": 17.13, "ce": 0.047227779432432726, "kl": 0.03851103741908446, "context": 0.06648329860763624}

runs/fixed_internvl/context/Middle8_s13/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Middle8_s13/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.14.mlp.down_proj": 144,
+    "model.language_model.layers.14.mlp.gate_proj": 144,
+    "model.language_model.layers.14.mlp.up_proj": 144,
+    "model.language_model.layers.14.self_attn.k_proj": 144,
+    "model.language_model.layers.14.self_attn.o_proj": 144,
+    "model.language_model.layers.14.self_attn.q_proj": 144,
+    "model.language_model.layers.14.self_attn.v_proj": 144,
+    "model.language_model.layers.15.mlp.down_proj": 144,
+    "model.language_model.layers.15.mlp.gate_proj": 144,
+    "model.language_model.layers.15.mlp.up_proj": 144,
+    "model.language_model.layers.15.self_attn.k_proj": 144,
+    "model.language_model.layers.15.self_attn.o_proj": 144,
+    "model.language_model.layers.15.self_attn.q_proj": 144,
+    "model.language_model.layers.15.self_attn.v_proj": 144,
+    "model.language_model.layers.16.mlp.down_proj": 144,
+    "model.language_model.layers.16.mlp.gate_proj": 144,
+    "model.language_model.layers.16.mlp.up_proj": 144,
+    "model.language_model.layers.16.self_attn.k_proj": 144,
+    "model.language_model.layers.16.self_attn.o_proj": 144,
+    "model.language_model.layers.16.self_attn.q_proj": 144,
+    "model.language_model.layers.16.self_attn.v_proj": 144,
+    "model.language_model.layers.17.mlp.down_proj": 144,
+    "model.language_model.layers.17.mlp.gate_proj": 144,
+    "model.language_model.layers.17.mlp.up_proj": 144,
+    "model.language_model.layers.17.self_attn.k_proj": 144,
+    "model.language_model.layers.17.self_attn.o_proj": 144,
+    "model.language_model.layers.17.self_attn.q_proj": 144,
+    "model.language_model.layers.17.self_attn.v_proj": 144,
+    "model.language_model.layers.18.mlp.down_proj": 144,
+    "model.language_model.layers.18.mlp.gate_proj": 144,
+    "model.language_model.layers.18.mlp.up_proj": 144,
+    "model.language_model.layers.18.self_attn.k_proj": 144,
+    "model.language_model.layers.18.self_attn.o_proj": 144,
+    "model.language_model.layers.18.self_attn.q_proj": 144,
+    "model.language_model.layers.18.self_attn.v_proj": 144,
+    "model.language_model.layers.19.mlp.down_proj": 144,
+    "model.language_model.layers.19.mlp.gate_proj": 144,
+    "model.language_model.layers.19.mlp.up_proj": 144,
+    "model.language_model.layers.19.self_attn.k_proj": 144,
+    "model.language_model.layers.19.self_attn.o_proj": 144,
+    "model.language_model.layers.19.self_attn.q_proj": 144,
+    "model.language_model.layers.19.self_attn.v_proj": 144,
+    "model.language_model.layers.20.mlp.down_proj": 144,
+    "model.language_model.layers.20.mlp.gate_proj": 144,
+    "model.language_model.layers.20.mlp.up_proj": 144,
+    "model.language_model.layers.20.self_attn.k_proj": 144,
+    "model.language_model.layers.20.self_attn.o_proj": 144,
+    "model.language_model.layers.20.self_attn.q_proj": 144,
+    "model.language_model.layers.20.self_attn.v_proj": 144,
+    "model.language_model.layers.21.mlp.down_proj": 144,
+    "model.language_model.layers.21.mlp.gate_proj": 144,
+    "model.language_model.layers.21.mlp.up_proj": 144,
+    "model.language_model.layers.21.self_attn.k_proj": 144,
+    "model.language_model.layers.21.self_attn.o_proj": 144,
+    "model.language_model.layers.21.self_attn.q_proj": 144,
+    "model.language_model.layers.21.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.14.mlp.down_proj": 72,
+    "model.language_model.layers.14.mlp.gate_proj": 72,
+    "model.language_model.layers.14.mlp.up_proj": 72,
+    "model.language_model.layers.14.self_attn.k_proj": 72,
+    "model.language_model.layers.14.self_attn.o_proj": 72,
+    "model.language_model.layers.14.self_attn.q_proj": 72,
+    "model.language_model.layers.14.self_attn.v_proj": 72,
+    "model.language_model.layers.15.mlp.down_proj": 72,
+    "model.language_model.layers.15.mlp.gate_proj": 72,
+    "model.language_model.layers.15.mlp.up_proj": 72,
+    "model.language_model.layers.15.self_attn.k_proj": 72,
+    "model.language_model.layers.15.self_attn.o_proj": 72,
+    "model.language_model.layers.15.self_attn.q_proj": 72,
+    "model.language_model.layers.15.self_attn.v_proj": 72,
+    "model.language_model.layers.16.mlp.down_proj": 72,
+    "model.language_model.layers.16.mlp.gate_proj": 72,
+    "model.language_model.layers.16.mlp.up_proj": 72,
+    "model.language_model.layers.16.self_attn.k_proj": 72,
+    "model.language_model.layers.16.self_attn.o_proj": 72,
+    "model.language_model.layers.16.self_attn.q_proj": 72,
+    "model.language_model.layers.16.self_attn.v_proj": 72,
+    "model.language_model.layers.17.mlp.down_proj": 72,
+    "model.language_model.layers.17.mlp.gate_proj": 72,
+    "model.language_model.layers.17.mlp.up_proj": 72,
+    "model.language_model.layers.17.self_attn.k_proj": 72,
+    "model.language_model.layers.17.self_attn.o_proj": 72,
+    "model.language_model.layers.17.self_attn.q_proj": 72,
+    "model.language_model.layers.17.self_attn.v_proj": 72,
+    "model.language_model.layers.18.mlp.down_proj": 72,
+    "model.language_model.layers.18.mlp.gate_proj": 72,
+    "model.language_model.layers.18.mlp.up_proj": 72,
+    "model.language_model.layers.18.self_attn.k_proj": 72,
+    "model.language_model.layers.18.self_attn.o_proj": 72,
+    "model.language_model.layers.18.self_attn.q_proj": 72,
+    "model.language_model.layers.18.self_attn.v_proj": 72,
+    "model.language_model.layers.19.mlp.down_proj": 72,
+    "model.language_model.layers.19.mlp.gate_proj": 72,
+    "model.language_model.layers.19.mlp.up_proj": 72,
+    "model.language_model.layers.19.self_attn.k_proj": 72,
+    "model.language_model.layers.19.self_attn.o_proj": 72,
+    "model.language_model.layers.19.self_attn.q_proj": 72,
+    "model.language_model.layers.19.self_attn.v_proj": 72,
+    "model.language_model.layers.20.mlp.down_proj": 72,
+    "model.language_model.layers.20.mlp.gate_proj": 72,
+    "model.language_model.layers.20.mlp.up_proj": 72,
+    "model.language_model.layers.20.self_attn.k_proj": 72,
+    "model.language_model.layers.20.self_attn.o_proj": 72,
+    "model.language_model.layers.20.self_attn.q_proj": 72,
+    "model.language_model.layers.20.self_attn.v_proj": 72,
+    "model.language_model.layers.21.mlp.down_proj": 72,
+    "model.language_model.layers.21.mlp.gate_proj": 72,
+    "model.language_model.layers.21.mlp.up_proj": 72,
+    "model.language_model.layers.21.self_attn.k_proj": 72,
+    "model.language_model.layers.21.self_attn.o_proj": 72,
+    "model.language_model.layers.21.self_attn.q_proj": 72,
+    "model.language_model.layers.21.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.17.self_attn.v_proj",
+    "model.language_model.layers.14.self_attn.o_proj",
+    "model.language_model.layers.14.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.14.self_attn.v_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.14.mlp.gate_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.14.self_attn.k_proj",
+    "model.language_model.layers.14.mlp.up_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.17.self_attn.o_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.14.self_attn.q_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.16.self_attn.k_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Middle8_s13/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 13,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Middle8",
+    "layers": [
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21
+    ],
+    "ranks": {
+      "14": 72,
+      "15": 72,
+      "16": 72,
+      "17": 72,
+      "18": 72,
+      "19": 72,
+      "20": 72,
+      "21": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Middle8_s13/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 7.267666339874268, "loss_scale": 4.0, "retries": 0, "seconds": 2.4, "max_memory_gb": 16.79, "ce": 0.5227766726748087, "kl": 0.08654405153356493, "context": 0.5660487064160407}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 9.985881805419922, "loss_scale": 4.0, "retries": 0, "seconds": 3.9, "max_memory_gb": 17.1, "ce": 0.6324257375890738, "kl": 0.4670185166748979, "context": 0.8659349749141256}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 23.663522720336914, "loss_scale": 4.0, "retries": 0, "seconds": 5.1, "max_memory_gb": 17.1, "ce": 0.879268079996109, "kl": 0.07087007910013199, "context": 0.9147031009197235}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 6.27983283996582, "loss_scale": 4.0, "retries": 0, "seconds": 6.4, "max_memory_gb": 17.1, "ce": 0.17109270740911597, "kl": 0.4704503118412049, "context": 0.40631786295125494}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 0.27986592054367065, "loss_scale": 4.0, "retries": 0, "seconds": 7.7, "max_memory_gb": 17.1, "ce": 0.009336486782558495, "kl": 0.008643434921395965, "context": 0.013658204316016054}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 4.2984442710876465, "loss_scale": 4.0, "retries": 0, "seconds": 9.0, "max_memory_gb": 17.1, "ce": 0.12388594840012956, "kl": 0.08798256940644933, "context": 0.16787723590095993}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 1.1457139253616333, "loss_scale": 4.0, "retries": 0, "seconds": 10.2, "max_memory_gb": 17.1, "ce": 0.04007169152646384, "kl": 0.06863953375659548, "context": 0.07439145805619773}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 0.8469809889793396, "loss_scale": 4.0, "retries": 0, "seconds": 11.5, "max_memory_gb": 17.1, "ce": 0.010966045781970024, "kl": 0.19999261040356942, "context": 0.11096234992146492}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 18.369144439697266, "loss_scale": 4.0, "retries": 0, "seconds": 12.8, "max_memory_gb": 17.1, "ce": 0.14033910539001226, "kl": 0.25545427680481225, "context": 0.26806624233722687}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 0.3725630044937134, "loss_scale": 4.0, "retries": 0, "seconds": 14.1, "max_memory_gb": 17.11, "ce": 0.0031311473867390305, "kl": 0.009847533161519095, "context": 0.008054914127569646}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.11181160807609558, "loss_scale": 4.0, "retries": 0, "seconds": 15.3, "max_memory_gb": 17.11, "ce": 0.0029340732871787623, "kl": 0.004157867609592358, "context": 0.005013006917579332}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 0.16662217676639557, "loss_scale": 4.0, "retries": 0, "seconds": 16.5, "max_memory_gb": 17.11, "ce": 0.005801972292829305, "kl": 0.002387453021356123, "context": 0.006995698861828714}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.01699841581285, "loss_scale": 4.0, "retries": 0, "seconds": 17.9, "max_memory_gb": 17.11, "ce": 0.0006282201666181209, "kl": 0.00014512293364532525, "context": 0.0007007816275290679}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 2.312485933303833, "loss_scale": 4.0, "retries": 0, "seconds": 19.1, "max_memory_gb": 17.11, "ce": 0.019269929704023525, "kl": 0.7789372908882797, "context": 0.40873857168480754}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 0.997968852519989, "loss_scale": 4.0, "retries": 0, "seconds": 20.3, "max_memory_gb": 17.11, "ce": 0.008817538405708092, "kl": 0.35184019571435243, "context": 0.18473764007546833}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 0.02769722044467926, "loss_scale": 4.0, "retries": 0, "seconds": 21.7, "max_memory_gb": 17.11, "ce": 0.001364095101962448, "kl": 0.0004551943762010069, "context": 0.001591692254805821}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 2.693572998046875, "loss_scale": 4.0, "retries": 0, "seconds": 22.9, "max_memory_gb": 17.11, "ce": 0.02266799451535917, "kl": 0.7107164640128758, "context": 0.37802621720766183}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.8201010227203369, "loss_scale": 4.0, "retries": 0, "seconds": 24.1, "max_memory_gb": 17.11, "ce": 0.008703848256118363, "kl": 0.26189105537378055, "context": 0.13964937972195912}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.467008113861084, "loss_scale": 4.0, "retries": 0, "seconds": 25.4, "max_memory_gb": 17.11, "ce": 0.004544342937151669, "kl": 0.10237005451926962, "context": 0.05572937160832225}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.10176992416381836, "loss_scale": 4.0, "retries": 0, "seconds": 26.7, "max_memory_gb": 17.11, "ce": 0.0007278399480128428, "kl": 0.005219292417677934, "context": 0.003337486257805722}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.0608484111726284, "loss_scale": 4.0, "retries": 0, "seconds": 27.9, "max_memory_gb": 17.11, "ce": 0.0009925905251293443, "kl": 0.004939144985769417, "context": 0.003462162901996635}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 5.315067291259766, "loss_scale": 4.0, "retries": 0, "seconds": 29.3, "max_memory_gb": 17.11, "ce": 0.0641442092746729, "kl": 0.16231555583544832, "context": 0.1453019905166002}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 1.9153435230255127, "loss_scale": 4.0, "retries": 0, "seconds": 30.5, "max_memory_gb": 17.11, "ce": 0.0009344172040073317, "kl": 0.3355297959433301, "context": 0.168699320003725}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.7421463131904602, "loss_scale": 4.0, "retries": 0, "seconds": 31.7, "max_memory_gb": 17.11, "ce": 0.011436271175625734, "kl": 0.07196349458899931, "context": 0.047418018424650654}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 2.6340513229370117, "loss_scale": 4.0, "retries": 0, "seconds": 33.1, "max_memory_gb": 17.11, "ce": 0.2201648182963254, "kl": 0.029799711890518665, "context": 0.23506467533297837}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 1.1853182315826416, "loss_scale": 4.0, "retries": 0, "seconds": 34.3, "max_memory_gb": 17.11, "ce": 0.0635497263428988, "kl": 0.44845601973793237, "context": 0.28777775121852756}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 1.2171905040740967, "loss_scale": 4.0, "retries": 0, "seconds": 35.7, "max_memory_gb": 17.11, "ce": 0.0785678444954101, "kl": 0.2615029815642629, "context": 0.20931933110114187}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 1.1953850984573364, "loss_scale": 4.0, "retries": 0, "seconds": 36.9, "max_memory_gb": 17.11, "ce": 0.14104485484131146, "kl": 0.15151943289674819, "context": 0.21680457051843405}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 3.8760664463043213, "loss_scale": 4.0, "retries": 0, "seconds": 38.1, "max_memory_gb": 17.11, "ce": 0.43889986420981586, "kl": 0.10677737282821909, "context": 0.4922885417472571}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.24171672761440277, "loss_scale": 4.0, "retries": 0, "seconds": 39.4, "max_memory_gb": 17.11, "ce": 0.008008737699128687, "kl": 0.00618995692639146, "context": 0.011103716242359951}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 7.659195423126221, "loss_scale": 4.0, "retries": 0, "seconds": 40.6, "max_memory_gb": 17.11, "ce": 0.7385283983312547, "kl": 0.08637947682291269, "context": 0.781718143960461}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 1.609564185142517, "loss_scale": 4.0, "retries": 0, "seconds": 41.8, "max_memory_gb": 17.11, "ce": 0.06697373976930976, "kl": 0.04046459318487905, "context": 0.08720603631809354}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 4.853436470031738, "loss_scale": 4.0, "retries": 0, "seconds": 43.1, "max_memory_gb": 17.11, "ce": 0.3915168778039515, "kl": 0.17692858836380765, "context": 0.47998118377290666}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.2578628063201904, "loss_scale": 4.0, "retries": 0, "seconds": 44.3, "max_memory_gb": 17.11, "ce": 0.012206086656078696, "kl": 0.6348959056776948, "context": 0.329654032131657}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.18597422540187836, "loss_scale": 4.0, "retries": 0, "seconds": 45.7, "max_memory_gb": 17.11, "ce": 0.00643701630178839, "kl": 0.0014087409363128245, "context": 0.007141387090086937}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 1.3376344442367554, "loss_scale": 4.0, "retries": 0, "seconds": 46.9, "max_memory_gb": 17.11, "ce": 0.18457478214986622, "kl": 0.02616096253041178, "context": 0.19765526277478784}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 1.2464311122894287, "loss_scale": 4.0, "retries": 0, "seconds": 48.2, "max_memory_gb": 17.11, "ce": 0.134739511879161, "kl": 0.27005270158406347, "context": 0.26976585830561817}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.8318137526512146, "loss_scale": 4.0, "retries": 0, "seconds": 49.5, "max_memory_gb": 17.11, "ce": 0.10245539317838848, "kl": 0.3311113713934901, "context": 0.26801107288338244}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.4140814542770386, "loss_scale": 4.0, "retries": 0, "seconds": 50.7, "max_memory_gb": 17.11, "ce": 0.07435933063970879, "kl": 0.008339872991200536, "context": 0.07852926373016089}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.33074814081192017, "loss_scale": 4.0, "retries": 0, "seconds": 51.9, "max_memory_gb": 17.11, "ce": 0.024376858258619905, "kl": 0.011650457279756665, "context": 0.030202086782082915}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 3.5774333477020264, "loss_scale": 4.0, "retries": 0, "seconds": 53.2, "max_memory_gb": 17.11, "ce": 0.419682931387797, "kl": 0.0671549232210964, "context": 0.453260388225317}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 0.10220687836408615, "loss_scale": 4.0, "retries": 0, "seconds": 54.4, "max_memory_gb": 17.11, "ce": 0.009720416099298745, "kl": 0.0004588693664118182, "context": 0.009949851024430245}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 0.11669468879699707, "loss_scale": 4.0, "retries": 0, "seconds": 55.6, "max_memory_gb": 17.11, "ce": 0.00904496927978471, "kl": 0.0019849682503263466, "context": 0.010037453495897353}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 14.612554550170898, "loss_scale": 4.0, "retries": 0, "seconds": 56.9, "max_memory_gb": 17.11, "ce": 0.3319702001172118, "kl": 0.868892596263322, "context": 0.7664165201131254}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 3.785691261291504, "loss_scale": 4.0, "retries": 0, "seconds": 58.1, "max_memory_gb": 17.11, "ce": 0.5521883111214265, "kl": 0.1266534779279027, "context": 0.6155150585109368}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 2.285661458969116, "loss_scale": 4.0, "retries": 0, "seconds": 59.4, "max_memory_gb": 17.11, "ce": 0.06200129759963602, "kl": 0.1173554421402514, "context": 0.12067902274429798}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 1.0178450345993042, "loss_scale": 4.0, "retries": 0, "seconds": 60.6, "max_memory_gb": 17.11, "ce": 0.021022088563768193, "kl": 0.06882073055021465, "context": 0.05543245328590274}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.07613334059715271, "loss_scale": 4.0, "retries": 0, "seconds": 61.8, "max_memory_gb": 17.11, "ce": 0.004641302541131154, "kl": 0.0025771285872906446, "context": 0.005929866922087967}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.33116093277931213, "loss_scale": 4.0, "retries": 0, "seconds": 63.1, "max_memory_gb": 17.11, "ce": 0.014649234275566414, "kl": 0.05171322915566634, "context": 0.04050584792275913}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.457086980342865, "loss_scale": 4.0, "retries": 0, "seconds": 64.3, "max_memory_gb": 17.11, "ce": 0.06603835505666211, "kl": 0.010125941858859733, "context": 0.07110132236266509}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 5.1885576248168945, "loss_scale": 4.0, "retries": 0, "seconds": 65.5, "max_memory_gb": 17.11, "ce": 0.12528954731533304, "kl": 0.012547569155231031, "context": 0.131563334929524}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.30356723070144653, "loss_scale": 4.0, "retries": 0, "seconds": 66.8, "max_memory_gb": 17.11, "ce": 0.020475167664699256, "kl": 0.09694078448228538, "context": 0.06894555990584195}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 1.3372509479522705, "loss_scale": 4.0, "retries": 0, "seconds": 68.0, "max_memory_gb": 17.11, "ce": 0.049560226732864976, "kl": 0.23505033878609538, "context": 0.16708539333194494}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 0.6955453753471375, "loss_scale": 4.0, "retries": 0, "seconds": 69.3, "max_memory_gb": 17.11, "ce": 0.06106608308618888, "kl": 0.03831193232326768, "context": 0.08022204897133633}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 0.5709947943687439, "loss_scale": 4.0, "retries": 0, "seconds": 70.5, "max_memory_gb": 17.11, "ce": 0.03320421323587652, "kl": 0.019773444259953976, "context": 0.043090935301734135}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.4611012041568756, "loss_scale": 4.0, "retries": 0, "seconds": 71.7, "max_memory_gb": 17.11, "ce": 0.030403361772187054, "kl": 0.07358132244553417, "context": 0.06719402503222227}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.09337354451417923, "loss_scale": 4.0, "retries": 0, "seconds": 72.9, "max_memory_gb": 17.11, "ce": 0.005889590902370401, "kl": 0.0063978184371080715, "context": 0.009088500373763964}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.01539679802954197, "loss_scale": 4.0, "retries": 0, "seconds": 74.2, "max_memory_gb": 17.11, "ce": 0.0006726852807332762, "kl": 0.00027907054571585377, "context": 0.0008122205690597184}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 2.9183032512664795, "loss_scale": 4.0, "retries": 0, "seconds": 75.4, "max_memory_gb": 17.11, "ce": 0.22670893142640125, "kl": 0.0627275438655488, "context": 0.25807270428776974}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 0.528854250907898, "loss_scale": 4.0, "retries": 0, "seconds": 76.7, "max_memory_gb": 17.11, "ce": 0.07533025793964043, "kl": 0.047270181552448776, "context": 0.09896534883591812}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 2.4541478157043457, "loss_scale": 4.0, "retries": 0, "seconds": 77.9, "max_memory_gb": 17.11, "ce": 0.20016034680884331, "kl": 0.3695092505208777, "context": 0.3849149724992458}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.33723703026771545, "loss_scale": 4.0, "retries": 0, "seconds": 79.1, "max_memory_gb": 17.11, "ce": 0.025655568460933864, "kl": 0.008038181280426215, "context": 0.029674659541342407}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 0.9826305508613586, "loss_scale": 4.0, "retries": 0, "seconds": 80.4, "max_memory_gb": 17.11, "ce": 0.06744895369774895, "kl": 0.1296767254716542, "context": 0.13228731602430344}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.13538554310798645, "loss_scale": 4.0, "retries": 0, "seconds": 81.6, "max_memory_gb": 17.11, "ce": 0.00720252949395217, "kl": 0.006155922543257475, "context": 0.010280490823788568}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 0.7613710761070251, "loss_scale": 4.0, "retries": 0, "seconds": 82.8, "max_memory_gb": 17.11, "ce": 0.0778461352456361, "kl": 0.12340254383161664, "context": 0.13954740576446056}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 1.6779775619506836, "loss_scale": 4.0, "retries": 0, "seconds": 84.2, "max_memory_gb": 17.11, "ce": 0.14817758916888124, "kl": 0.0784205752967182, "context": 0.18738787684424096}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.046560484915971756, "loss_scale": 4.0, "retries": 0, "seconds": 85.4, "max_memory_gb": 17.11, "ce": 0.0015411510471494694, "kl": 0.0043086975574624375, "context": 0.003695499902278243}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.4010637104511261, "loss_scale": 4.0, "retries": 0, "seconds": 86.8, "max_memory_gb": 17.11, "ce": 0.006040393575858616, "kl": 0.020363369014376076, "context": 0.016222078330429213}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 4.571075916290283, "loss_scale": 4.0, "retries": 0, "seconds": 88.0, "max_memory_gb": 17.11, "ce": 0.6827531609960715, "kl": 0.014203698134224396, "context": 0.6898550300829811}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.5724844932556152, "loss_scale": 4.0, "retries": 0, "seconds": 89.3, "max_memory_gb": 17.11, "ce": 0.005278242673284694, "kl": 0.0017183919167109707, "context": 0.006137438510904758}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.025203850120306015, "loss_scale": 4.0, "retries": 0, "seconds": 90.6, "max_memory_gb": 17.11, "ce": 0.0005465840686156298, "kl": 0.0019861169457726646, "context": 0.0015396425151266158}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.9401121139526367, "loss_scale": 4.0, "retries": 0, "seconds": 91.8, "max_memory_gb": 17.11, "ce": 0.006025346485785121, "kl": 0.5055556148290634, "context": 0.2588031585000863}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 3.5871641635894775, "loss_scale": 4.0, "retries": 0, "seconds": 93.0, "max_memory_gb": 17.11, "ce": 0.20185987207150902, "kl": 0.053332029116063495, "context": 0.22852588104797178}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.7856667637825012, "loss_scale": 4.0, "retries": 0, "seconds": 94.3, "max_memory_gb": 17.11, "ce": 0.060821459070666606, "kl": 0.09876591577574345, "context": 0.11020442067683689}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 3.01835036277771, "loss_scale": 4.0, "retries": 0, "seconds": 95.5, "max_memory_gb": 17.11, "ce": 0.46855875216442655, "kl": 1.0183786083944142, "context": 0.9777480470947921}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 2.8554112911224365, "loss_scale": 4.0, "retries": 0, "seconds": 96.7, "max_memory_gb": 17.11, "ce": 0.018984415236445784, "kl": 1.6664965889649466, "context": 0.8522327337414026}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.20266038179397583, "loss_scale": 4.0, "retries": 0, "seconds": 98.0, "max_memory_gb": 17.11, "ce": 0.013569262548116967, "kl": 0.006258797220652923, "context": 0.016698661202099174}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.4441603422164917, "loss_scale": 4.0, "retries": 0, "seconds": 99.3, "max_memory_gb": 17.11, "ce": 0.001937719756824663, "kl": 0.09749817227566382, "context": 0.050686806127487216}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.6947498321533203, "loss_scale": 4.0, "retries": 0, "seconds": 100.6, "max_memory_gb": 17.11, "ce": 0.01370332315491396, "kl": 0.020033590182720218, "context": 0.0237201182753779}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.1776585727930069, "loss_scale": 4.0, "retries": 0, "seconds": 101.8, "max_memory_gb": 17.11, "ce": 0.0013153123545635026, "kl": 0.02520649682264775, "context": 0.013918561169703025}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 1.0282574892044067, "loss_scale": 4.0, "retries": 0, "seconds": 103.0, "max_memory_gb": 17.11, "ce": 0.09307228005491197, "kl": 0.045693907770328224, "context": 0.11591923353262246}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.15996363759040833, "loss_scale": 4.0, "retries": 0, "seconds": 104.2, "max_memory_gb": 17.11, "ce": 0.010024275747127831, "kl": 0.0033499338787805755, "context": 0.011699242750182748}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.543251872062683, "loss_scale": 4.0, "retries": 0, "seconds": 105.4, "max_memory_gb": 17.11, "ce": 0.11194520170829492, "kl": 0.4436582204871229, "context": 0.33377430665132124}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 0.9987830519676208, "loss_scale": 4.0, "retries": 0, "seconds": 106.6, "max_memory_gb": 17.11, "ce": 0.07303838026928133, "kl": 0.025772095626507507, "context": 0.08592443145971629}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.18876773118972778, "loss_scale": 4.0, "retries": 0, "seconds": 107.9, "max_memory_gb": 17.11, "ce": 0.0032705061148590175, "kl": 0.000874454226789112, "context": 0.0037077333145134617}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 1.6453927755355835, "loss_scale": 4.0, "retries": 0, "seconds": 109.1, "max_memory_gb": 17.11, "ce": 0.03700982689042576, "kl": 0.15723723458359018, "context": 0.11562844816944562}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 1.4381920099258423, "loss_scale": 4.0, "retries": 0, "seconds": 110.2, "max_memory_gb": 17.11, "ce": 0.006143941718619317, "kl": 0.41465694224461913, "context": 0.2134724142961204}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.37185966968536377, "loss_scale": 4.0, "retries": 0, "seconds": 111.5, "max_memory_gb": 17.11, "ce": 0.01613235419790726, "kl": 0.002299874281817438, "context": 0.01728229103173362}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.40081989765167236, "loss_scale": 4.0, "retries": 0, "seconds": 112.7, "max_memory_gb": 17.11, "ce": 0.020623237352992874, "kl": 0.015212930033158045, "context": 0.028229702485987218}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.3535056710243225, "loss_scale": 4.0, "retries": 0, "seconds": 113.8, "max_memory_gb": 17.11, "ce": 0.01107393013080582, "kl": 0.0054540318888030015, "context": 0.013800946064293385}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 4.25390625, "loss_scale": 4.0, "retries": 0, "seconds": 115.1, "max_memory_gb": 17.11, "ce": 0.5167119475372601, "kl": 0.20410182525665732, "context": 0.6187628541083541}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 4.189716339111328, "loss_scale": 4.0, "retries": 0, "seconds": 116.3, "max_memory_gb": 17.11, "ce": 0.0701744915713789, "kl": 0.1336536529197474, "context": 0.13700131853693165}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 4.299241542816162, "loss_scale": 4.0, "retries": 0, "seconds": 117.6, "max_memory_gb": 17.11, "ce": 0.05693178980436642, "kl": 0.23712499170505907, "context": 0.17549428407801315}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 1.2540346384048462, "loss_scale": 4.0, "retries": 0, "seconds": 118.8, "max_memory_gb": 17.11, "ce": 0.03412012170883827, "kl": 0.01637106214184314, "context": 0.042305654351366684}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 5.216135025024414, "loss_scale": 4.0, "retries": 0, "seconds": 120.0, "max_memory_gb": 17.11, "ce": 0.5961256725349813, "kl": 0.009414028259925544, "context": 0.6008326958399266}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 3.867357015609741, "loss_scale": 4.0, "retries": 0, "seconds": 121.3, "max_memory_gb": 17.11, "ce": 0.23370105047069956, "kl": 0.013857151523552602, "context": 0.2406296243716497}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.16834264993667603, "loss_scale": 4.0, "retries": 0, "seconds": 122.5, "max_memory_gb": 17.11, "ce": 0.013343721293495037, "kl": 0.006914992933161557, "context": 0.01680121710523963}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.23808762431144714, "loss_scale": 4.0, "retries": 0, "seconds": 123.6, "max_memory_gb": 17.11, "ce": 0.0031383355817524716, "kl": 0.06579979190792073, "context": 0.03603823199227918}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.7789368629455566, "loss_scale": 4.0, "retries": 0, "seconds": 124.9, "max_memory_gb": 17.11, "ce": 0.030238637496950105, "kl": 0.004934275115374476, "context": 0.03270577524381224}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.4944862425327301, "loss_scale": 4.0, "retries": 0, "seconds": 126.1, "max_memory_gb": 17.11, "ce": 0.04470097647572402, "kl": 0.23258898797939764, "context": 0.1609954719315283}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.1288250833749771, "loss_scale": 4.0, "retries": 0, "seconds": 127.5, "max_memory_gb": 17.11, "ce": 0.004712162452051416, "kl": 0.012190129607915878, "context": 0.010807227110490203}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.3385027348995209, "loss_scale": 4.0, "retries": 0, "seconds": 128.7, "max_memory_gb": 17.11, "ce": 0.04168182669673115, "kl": 0.013332747941603884, "context": 0.04834820213727653}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.13979250192642212, "loss_scale": 4.0, "retries": 0, "seconds": 129.9, "max_memory_gb": 17.11, "ce": 0.01276246047927998, "kl": 0.0047490331508015515, "context": 0.015136977410293184}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.5058029890060425, "loss_scale": 4.0, "retries": 0, "seconds": 131.1, "max_memory_gb": 17.11, "ce": 0.01805279596010223, "kl": 0.00920934540226881, "context": 0.02265746967168525}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 0.8901170492172241, "loss_scale": 4.0, "retries": 0, "seconds": 132.4, "max_memory_gb": 17.11, "ce": 0.017289377108681947, "kl": 0.29609872883884236, "context": 0.16533874266315252}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.03705047443509102, "loss_scale": 4.0, "retries": 0, "seconds": 133.5, "max_memory_gb": 17.11, "ce": 0.0030989819206297398, "kl": 0.0007993325416464359, "context": 0.003498648147797212}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 1.013035535812378, "loss_scale": 4.0, "retries": 0, "seconds": 134.9, "max_memory_gb": 17.11, "ce": 0.013780894281808287, "kl": 0.20210995159685297, "context": 0.11483587096154224}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 2.2875678539276123, "loss_scale": 4.0, "retries": 0, "seconds": 136.1, "max_memory_gb": 17.11, "ce": 0.47343580971937627, "kl": 0.0015089671028363227, "context": 0.4741902838286478}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 4.233500003814697, "loss_scale": 4.0, "retries": 0, "seconds": 137.2, "max_memory_gb": 17.11, "ce": 0.33605915572843514, "kl": 0.17117877025157213, "context": 0.42164855217561126}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.22235344350337982, "loss_scale": 4.0, "retries": 0, "seconds": 138.5, "max_memory_gb": 17.11, "ce": 0.023933976888656616, "kl": 0.002601328887976706, "context": 0.02523464197292924}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.024017736315727234, "loss_scale": 4.0, "retries": 0, "seconds": 139.8, "max_memory_gb": 17.11, "ce": 0.0016116350016091019, "kl": 0.0009209239906340372, "context": 0.002072096976917237}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.3627607524394989, "loss_scale": 4.0, "retries": 0, "seconds": 141.1, "max_memory_gb": 17.11, "ce": 0.02495834353612736, "kl": 0.009561857594235335, "context": 0.029739271849393845}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 3.7256112098693848, "loss_scale": 4.0, "retries": 0, "seconds": 142.3, "max_memory_gb": 17.11, "ce": 0.01223351527005434, "kl": 0.12893434590660036, "context": 0.0767006886890158}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 1.4334568977355957, "loss_scale": 4.0, "retries": 0, "seconds": 143.5, "max_memory_gb": 17.11, "ce": 0.022105860960436985, "kl": 0.26518902528187027, "context": 0.15470037441991735}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 1.8795194625854492, "loss_scale": 4.0, "retries": 0, "seconds": 144.8, "max_memory_gb": 17.11, "ce": 0.0808915946399793, "kl": 0.08431920224393252, "context": 0.1230511957546696}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 0.9761280417442322, "loss_scale": 4.0, "retries": 0, "seconds": 146.0, "max_memory_gb": 17.11, "ce": 0.09621806960785761, "kl": 0.27730639906530996, "context": 0.23487126165127847}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.17685532569885254, "loss_scale": 4.0, "retries": 0, "seconds": 147.1, "max_memory_gb": 17.11, "ce": 0.0074761060095625, "kl": 0.005320285909874656, "context": 0.010136249198694713}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.9116638898849487, "loss_scale": 4.0, "retries": 0, "seconds": 148.4, "max_memory_gb": 17.11, "ce": 0.20926952571608126, "kl": 0.005535639302252093, "context": 0.2120373378857039}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.09595958888530731, "loss_scale": 4.0, "retries": 0, "seconds": 149.6, "max_memory_gb": 17.11, "ce": 0.006956458906643093, "kl": 0.003610198291426059, "context": 0.008761558085097931}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 5.56441068649292, "loss_scale": 4.0, "retries": 0, "seconds": 150.9, "max_memory_gb": 17.11, "ce": 0.38481218717060983, "kl": 0.20000636148324702, "context": 0.4848153729690239}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.9337698221206665, "loss_scale": 4.0, "retries": 0, "seconds": 152.2, "max_memory_gb": 17.11, "ce": 0.0465789945301367, "kl": 0.04313072524746531, "context": 0.06814435715205036}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.0810900926589966, "loss_scale": 4.0, "retries": 0, "seconds": 153.4, "max_memory_gb": 17.11, "ce": 0.12474327202653512, "kl": 0.03099184091479401, "context": 0.14023919387545902}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.6585838198661804, "loss_scale": 4.0, "retries": 0, "seconds": 154.6, "max_memory_gb": 17.11, "ce": 0.06523059215396643, "kl": 0.0352653912268579, "context": 0.08286328706890345}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 3.442538022994995, "loss_scale": 4.0, "retries": 0, "seconds": 155.9, "max_memory_gb": 17.11, "ce": 0.547040409874171, "kl": 0.4386495268913677, "context": 0.7663651532930089}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.8834884762763977, "loss_scale": 4.0, "retries": 0, "seconds": 157.1, "max_memory_gb": 17.11, "ce": 0.10800605244003236, "kl": 0.08827718084285152, "context": 0.1521446438273415}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 3.2803361415863037, "loss_scale": 4.0, "retries": 0, "seconds": 158.5, "max_memory_gb": 17.11, "ce": 0.5301789967343211, "kl": 0.0458578635007143, "context": 0.5531079173088074}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.11931058019399643, "loss_scale": 4.0, "retries": 0, "seconds": 159.6, "max_memory_gb": 17.11, "ce": 0.009882983897114173, "kl": 0.0006564787981915288, "context": 0.01021122356178239}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 3.658745765686035, "loss_scale": 4.0, "retries": 0, "seconds": 160.8, "max_memory_gb": 17.11, "ce": 0.24940624088048935, "kl": 0.48953441900084727, "context": 0.49417346669360995}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 0.7041659951210022, "loss_scale": 4.0, "retries": 0, "seconds": 162.1, "max_memory_gb": 17.11, "ce": 0.10304500150959939, "kl": 0.03558636139769078, "context": 0.12083818385144696}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.49345314502716064, "loss_scale": 4.0, "retries": 0, "seconds": 163.3, "max_memory_gb": 17.11, "ce": 0.03438362351153046, "kl": 0.24740407924400643, "context": 0.15808567102067173}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 4.948169708251953, "loss_scale": 4.0, "retries": 0, "seconds": 164.5, "max_memory_gb": 17.11, "ce": 0.15215532016009092, "kl": 0.11413265159353614, "context": 0.20922164153307676}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.8763121962547302, "loss_scale": 4.0, "retries": 0, "seconds": 165.8, "max_memory_gb": 17.11, "ce": 0.13684907770948485, "kl": 0.0433218228390615, "context": 0.1585099890944548}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 1.2200323343276978, "loss_scale": 4.0, "retries": 0, "seconds": 167.0, "max_memory_gb": 17.11, "ce": 0.2046660985215567, "kl": 0.05090200016275048, "context": 0.2301170999999158}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 1.5406619310379028, "loss_scale": 4.0, "retries": 0, "seconds": 168.3, "max_memory_gb": 17.11, "ce": 0.2594305112725124, "kl": 0.004240723737893859, "context": 0.26155088664381765}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 0.9422113299369812, "loss_scale": 4.0, "retries": 0, "seconds": 169.5, "max_memory_gb": 17.11, "ce": 0.037820560653926805, "kl": 0.0691176514410472, "context": 0.0723793854995165}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 5.065408706665039, "loss_scale": 4.0, "retries": 0, "seconds": 170.7, "max_memory_gb": 17.11, "ce": 0.2350088789826259, "kl": 0.1496397694645566, "context": 0.30982876324560493}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 0.2353263944387436, "loss_scale": 4.0, "retries": 0, "seconds": 172.0, "max_memory_gb": 17.11, "ce": 0.01994873244257178, "kl": 0.009098797484512033, "context": 0.024498131606378593}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 4.948489665985107, "loss_scale": 4.0, "retries": 0, "seconds": 173.2, "max_memory_gb": 17.11, "ce": 0.15870562568306923, "kl": 0.1841671944130212, "context": 0.2507892190478742}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 1.2730512619018555, "loss_scale": 4.0, "retries": 0, "seconds": 174.5, "max_memory_gb": 17.11, "ce": 0.017684492166154087, "kl": 0.10815309362578773, "context": 0.07176103995152516}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.18402834236621857, "loss_scale": 4.0, "retries": 0, "seconds": 175.8, "max_memory_gb": 17.11, "ce": 0.01197301660431549, "kl": 0.003236806744553178, "context": 0.013591419556178153}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.16945569217205048, "loss_scale": 4.0, "retries": 0, "seconds": 177.0, "max_memory_gb": 17.11, "ce": 0.014996124999015592, "kl": 0.0007050771782814991, "context": 0.015348663815530017}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 2.0561671257019043, "loss_scale": 4.0, "retries": 0, "seconds": 178.2, "max_memory_gb": 17.11, "ce": 0.1101740647573024, "kl": 0.01658935359955649, "context": 0.1184687395580113}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 3.702110767364502, "loss_scale": 4.0, "retries": 0, "seconds": 179.6, "max_memory_gb": 17.11, "ce": 0.3775117821060121, "kl": 0.023243553026986774, "context": 0.38913356757257134}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 1.7970097064971924, "loss_scale": 4.0, "retries": 0, "seconds": 180.8, "max_memory_gb": 17.11, "ce": 0.04781146673485637, "kl": 0.1299145605880767, "context": 0.11276874458417296}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.031111285090446472, "loss_scale": 4.0, "retries": 0, "seconds": 182.1, "max_memory_gb": 17.11, "ce": 0.0015792519043316133, "kl": 0.0011068372405134141, "context": 0.0021326706118998118}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.2562897205352783, "loss_scale": 4.0, "retries": 0, "seconds": 183.2, "max_memory_gb": 17.11, "ce": 0.02262483439699281, "kl": 0.00846370429462695, "context": 0.026856685915845446}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.946480631828308, "loss_scale": 4.0, "retries": 0, "seconds": 184.4, "max_memory_gb": 17.11, "ce": 0.3428112466499442, "kl": 0.3000386537532904, "context": 0.4928305813809857}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.4944201707839966, "loss_scale": 4.0, "retries": 0, "seconds": 185.7, "max_memory_gb": 17.11, "ce": 0.015076017618412152, "kl": 0.024313731359143276, "context": 0.02723288320703432}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 0.38527730107307434, "loss_scale": 4.0, "retries": 0, "seconds": 187.0, "max_memory_gb": 17.11, "ce": 0.025373700074851513, "kl": 0.00521508719589292, "context": 0.02798124449327588}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.7289572358131409, "loss_scale": 4.0, "retries": 0, "seconds": 188.1, "max_memory_gb": 17.11, "ce": 0.08048678687919164, "kl": 0.014139177853394358, "context": 0.08755637397553073}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.7196829319000244, "loss_scale": 4.0, "retries": 0, "seconds": 189.5, "max_memory_gb": 17.11, "ce": 0.02978328987956047, "kl": 0.02908867668156745, "context": 0.044327628158498555}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 0.20780867338180542, "loss_scale": 4.0, "retries": 0, "seconds": 190.7, "max_memory_gb": 17.11, "ce": 0.0017183106028824113, "kl": 0.01702360704075545, "context": 0.010230114072328433}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.9164941310882568, "loss_scale": 4.0, "retries": 0, "seconds": 191.9, "max_memory_gb": 17.11, "ce": 0.005428451106126886, "kl": 0.4013118352277161, "context": 0.20608437058399431}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.3797259032726288, "loss_scale": 4.0, "retries": 0, "seconds": 193.2, "max_memory_gb": 17.11, "ce": 0.03250983557518339, "kl": 0.09816550824325532, "context": 0.08159258598607266}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.9873893857002258, "loss_scale": 4.0, "retries": 0, "seconds": 194.4, "max_memory_gb": 17.11, "ce": 0.1786922741594026, "kl": 0.015837172211831785, "context": 0.1866108580143191}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 1.7427477836608887, "loss_scale": 4.0, "retries": 0, "seconds": 195.5, "max_memory_gb": 17.11, "ce": 0.02241424535168335, "kl": 0.01946619628802182, "context": 0.032147343503311276}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.3002237379550934, "loss_scale": 4.0, "retries": 0, "seconds": 196.8, "max_memory_gb": 17.11, "ce": 0.028282884741201997, "kl": 0.024671495804795995, "context": 0.040618632454425097}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.97561514377594, "loss_scale": 4.0, "retries": 0, "seconds": 198.0, "max_memory_gb": 17.11, "ce": 0.19320565793896094, "kl": 0.11160797861521132, "context": 0.24900963972322643}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.8211116194725037, "loss_scale": 4.0, "retries": 0, "seconds": 199.5, "max_memory_gb": 17.11, "ce": 0.007857926422730088, "kl": 0.04018855250615161, "context": 0.027952201780863106}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 1.0787032842636108, "loss_scale": 4.0, "retries": 0, "seconds": 200.7, "max_memory_gb": 17.11, "ce": 0.10542229490238242, "kl": 0.16014581990020815, "context": 0.18549520533997566}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 2.45560884475708, "loss_scale": 4.0, "retries": 0, "seconds": 201.9, "max_memory_gb": 17.11, "ce": 0.2309348795679398, "kl": 0.15863271306079696, "context": 0.31025124358711764}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 1.6490504741668701, "loss_scale": 4.0, "retries": 0, "seconds": 203.3, "max_memory_gb": 17.11, "ce": 0.053524367744103074, "kl": 0.06042133212758927, "context": 0.08373503747861832}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 6.02878475189209, "loss_scale": 4.0, "retries": 0, "seconds": 204.6, "max_memory_gb": 17.11, "ce": 0.1274250484420918, "kl": 0.4210951642889995, "context": 0.33797261776635423}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 1.4170079231262207, "loss_scale": 4.0, "retries": 0, "seconds": 205.8, "max_memory_gb": 17.11, "ce": 0.09222357830731198, "kl": 0.04160312521162268, "context": 0.11302513649570756}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 1.0137735605239868, "loss_scale": 4.0, "retries": 0, "seconds": 207.2, "max_memory_gb": 17.11, "ce": 0.025010731333168224, "kl": 0.010442224025609903, "context": 0.030231842945795506}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 3.2155871391296387, "loss_scale": 4.0, "retries": 0, "seconds": 208.4, "max_memory_gb": 17.11, "ce": 0.32876535586547107, "kl": 0.045940873300423846, "context": 0.3517358038807288}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.39346784353256226, "loss_scale": 4.0, "retries": 0, "seconds": 209.6, "max_memory_gb": 17.11, "ce": 0.026732913102023304, "kl": 0.0050839299510698766, "context": 0.02927487762644887}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.11232098937034607, "loss_scale": 4.0, "retries": 0, "seconds": 210.6, "max_memory_gb": 17.11, "ce": 0.009801943975617178, "kl": 0.001921561991366616, "context": 0.010762724879896268}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 2.0044496059417725, "loss_scale": 4.0, "retries": 0, "seconds": 211.8, "max_memory_gb": 17.11, "ce": 0.08850903672282584, "kl": 0.9393175505101681, "context": 0.5581678189337254}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.023844759911298752, "loss_scale": 4.0, "retries": 0, "seconds": 213.1, "max_memory_gb": 17.11, "ce": 0.0013890900008846074, "kl": 0.00021247402673907345, "context": 0.0014953270147088915}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 1.0150810480117798, "loss_scale": 4.0, "retries": 0, "seconds": 214.3, "max_memory_gb": 17.11, "ce": 0.01570319102756912, "kl": 0.06766604370204732, "context": 0.04953621167805977}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.1913895308971405, "loss_scale": 4.0, "retries": 0, "seconds": 215.5, "max_memory_gb": 17.11, "ce": 0.008524031058186665, "kl": 0.015126122976653278, "context": 0.016087092604720965}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.1312771439552307, "loss_scale": 4.0, "retries": 0, "seconds": 216.8, "max_memory_gb": 17.11, "ce": 0.009917907795170322, "kl": 0.002423783007543534, "context": 0.011129798978799954}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.6971184015274048, "loss_scale": 4.0, "retries": 0, "seconds": 218.0, "max_memory_gb": 17.11, "ce": 0.04750833584694192, "kl": 0.0057200071073566505, "context": 0.05036834102065768}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 3.697138547897339, "loss_scale": 4.0, "retries": 0, "seconds": 219.2, "max_memory_gb": 17.11, "ce": 0.12398166139610112, "kl": 0.03594297473318875, "context": 0.14195315330289304}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.07898267358541489, "loss_scale": 4.0, "retries": 0, "seconds": 220.5, "max_memory_gb": 17.11, "ce": 0.005084288983198348, "kl": 0.0005147913980181329, "context": 0.005341684460290708}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.9306463003158569, "loss_scale": 4.0, "retries": 0, "seconds": 221.8, "max_memory_gb": 17.11, "ce": 0.08063827893784037, "kl": 0.5407693613797164, "context": 0.3510229484527372}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 1.9506157636642456, "loss_scale": 4.0, "retries": 0, "seconds": 223.1, "max_memory_gb": 17.11, "ce": 0.12108830641955137, "kl": 0.02595821034628898, "context": 0.1340674131643027}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.5700632333755493, "loss_scale": 4.0, "retries": 0, "seconds": 224.2, "max_memory_gb": 17.11, "ce": 0.3400413132912945, "kl": 0.017235954912393936, "context": 0.3486592813860625}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 1.0558565855026245, "loss_scale": 4.0, "retries": 0, "seconds": 225.3, "max_memory_gb": 17.11, "ce": 0.052595749555621296, "kl": 0.05206160229863599, "context": 0.0786265479400754}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 1.037811040878296, "loss_scale": 4.0, "retries": 0, "seconds": 226.7, "max_memory_gb": 17.11, "ce": 0.004832246864680201, "kl": 0.24250995821785182, "context": 0.1260872227139771}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.7415046095848083, "loss_scale": 4.0, "retries": 0, "seconds": 228.0, "max_memory_gb": 17.11, "ce": 0.06729384500067681, "kl": 0.19820659958259057, "context": 0.16639714478515089}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.0583258680999279, "loss_scale": 4.0, "retries": 0, "seconds": 229.2, "max_memory_gb": 17.11, "ce": 0.0037094849249115214, "kl": 0.000253408502203456, "context": 0.003836189236608334}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.19226542115211487, "loss_scale": 4.0, "retries": 0, "seconds": 230.5, "max_memory_gb": 17.11, "ce": 0.0051901981205446646, "kl": 0.007733023423497798, "context": 0.009056709634023719}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.38466328382492065, "loss_scale": 4.0, "retries": 0, "seconds": 231.7, "max_memory_gb": 17.11, "ce": 0.007451889505318832, "kl": 0.10603687197908584, "context": 0.0604703240969684}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 4.579848766326904, "loss_scale": 4.0, "retries": 0, "seconds": 232.9, "max_memory_gb": 17.11, "ce": 0.07744275849108817, "kl": 0.1254136366860621, "context": 0.1401495692844037}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 0.8109502196311951, "loss_scale": 4.0, "retries": 0, "seconds": 234.3, "max_memory_gb": 17.11, "ce": 0.016386537696234882, "kl": 0.6150111241659033, "context": 0.32389211098779924}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.8613340854644775, "loss_scale": 4.0, "retries": 0, "seconds": 235.5, "max_memory_gb": 17.11, "ce": 0.23562488085008226, "kl": 0.008667314632475609, "context": 0.23995853327505756}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 4.901997089385986, "loss_scale": 4.0, "retries": 0, "seconds": 236.7, "max_memory_gb": 17.11, "ce": 0.40528633902431466, "kl": 0.14307975003612228, "context": 0.4768262002908159}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.055499643087387085, "loss_scale": 4.0, "retries": 0, "seconds": 238.0, "max_memory_gb": 17.11, "ce": 0.002133863337803632, "kl": 0.0038594526158703957, "context": 0.0040635895857121795}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 3.9580752849578857, "loss_scale": 4.0, "retries": 0, "seconds": 239.0, "max_memory_gb": 17.11, "ce": 0.168783272310975, "kl": 0.15144800939015113, "context": 0.24450728425290436}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.6146224737167358, "loss_scale": 4.0, "retries": 0, "seconds": 240.2, "max_memory_gb": 17.11, "ce": 0.35390813380945474, "kl": 0.0451721062490833, "context": 0.3764941799454391}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.04346539452672005, "loss_scale": 4.0, "retries": 0, "seconds": 241.2, "max_memory_gb": 17.11, "ce": 0.0025035604485310614, "kl": 0.0010277596995820204, "context": 0.003017440321855247}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.8479408621788025, "loss_scale": 4.0, "retries": 0, "seconds": 242.4, "max_memory_gb": 17.11, "ce": 0.07310554489959031, "kl": 0.008534942506230436, "context": 0.07737301266752183}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.18724651634693146, "loss_scale": 4.0, "retries": 0, "seconds": 243.8, "max_memory_gb": 17.11, "ce": 0.017927635592059232, "kl": 0.0007103402822394855, "context": 0.018282805482158437}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.8866773843765259, "loss_scale": 4.0, "retries": 0, "seconds": 244.9, "max_memory_gb": 17.11, "ce": 0.0031285603981814347, "kl": 0.22540429678520013, "context": 0.11583070920460159}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 5.097170352935791, "loss_scale": 4.0, "retries": 0, "seconds": 246.0, "max_memory_gb": 17.11, "ce": 0.07387466140789911, "kl": 0.5553734894961053, "context": 0.35156139932223596}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.5978682041168213, "loss_scale": 4.0, "retries": 0, "seconds": 247.3, "max_memory_gb": 17.11, "ce": 0.07496901060221717, "kl": 0.005799855753139127, "context": 0.07786893544835038}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 0.8914554119110107, "loss_scale": 4.0, "retries": 0, "seconds": 248.5, "max_memory_gb": 17.11, "ce": 0.11324885362409987, "kl": 0.006646388763329014, "context": 0.11657204490620643}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.5012761354446411, "loss_scale": 4.0, "retries": 0, "seconds": 249.9, "max_memory_gb": 17.11, "ce": 0.25758902565576136, "kl": 0.03083117512142053, "context": 0.27300460438709706}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 1.476807713508606, "loss_scale": 4.0, "retries": 0, "seconds": 251.1, "max_memory_gb": 17.11, "ce": 0.06766855111345649, "kl": 0.07259927139239153, "context": 0.10396818682784215}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.04854991286993027, "loss_scale": 4.0, "retries": 0, "seconds": 252.2, "max_memory_gb": 17.11, "ce": 0.0031549091218039393, "kl": 0.002026209459245365, "context": 0.004168013896560296}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 2.1809844970703125, "loss_scale": 4.0, "retries": 0, "seconds": 253.6, "max_memory_gb": 17.11, "ce": 0.04351760740973987, "kl": 0.04537762509426102, "context": 0.06620642001507804}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.35524120926856995, "loss_scale": 4.0, "retries": 0, "seconds": 254.8, "max_memory_gb": 17.11, "ce": 0.032051264308393, "kl": 0.0028899977478431538, "context": 0.03349626378621906}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.9612207412719727, "loss_scale": 4.0, "retries": 0, "seconds": 256.0, "max_memory_gb": 17.11, "ce": 0.007353808963671327, "kl": 0.36978078117863333, "context": 0.19224419959937222}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.08577346056699753, "loss_scale": 4.0, "retries": 0, "seconds": 257.4, "max_memory_gb": 17.11, "ce": 0.004857622225244995, "kl": 0.001867675257017254, "context": 0.0057914598801289685}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 1.5678925514221191, "loss_scale": 4.0, "retries": 0, "seconds": 258.8, "max_memory_gb": 17.11, "ce": 0.08601699862629175, "kl": 0.08743233731365763, "context": 0.12973316363058984}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.04291004315018654, "loss_scale": 4.0, "retries": 0, "seconds": 260.1, "max_memory_gb": 17.11, "ce": 0.002803934970870614, "kl": 0.0010769684668048285, "context": 0.003342419207911007}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 1.524444580078125, "loss_scale": 4.0, "retries": 0, "seconds": 261.4, "max_memory_gb": 17.11, "ce": 0.20175180083606392, "kl": 0.05278641768381931, "context": 0.22814501193352044}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 1.897505521774292, "loss_scale": 4.0, "retries": 0, "seconds": 262.6, "max_memory_gb": 17.11, "ce": 0.48374994231562596, "kl": 0.12512880179201602, "context": 0.5463143731321907}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.6426997184753418, "loss_scale": 4.0, "retries": 0, "seconds": 264.0, "max_memory_gb": 17.11, "ce": 0.12552579751354642, "kl": 0.0580219933835906, "context": 0.15453679804340936}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 4.4022674560546875, "loss_scale": 4.0, "retries": 0, "seconds": 265.2, "max_memory_gb": 17.11, "ce": 0.10039673571009189, "kl": 0.8826572537363973, "context": 0.5417253494961187}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.9552062749862671, "loss_scale": 4.0, "retries": 0, "seconds": 266.4, "max_memory_gb": 17.11, "ce": 0.17278960505791474, "kl": 0.03474634062877158, "context": 0.19016278078197502}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.2138370275497437, "loss_scale": 4.0, "retries": 0, "seconds": 267.7, "max_memory_gb": 17.11, "ce": 0.2951449890679214, "kl": 0.056831850844901055, "context": 0.323560907272622}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 2.735226631164551, "loss_scale": 4.0, "retries": 0, "seconds": 268.9, "max_memory_gb": 17.11, "ce": 0.466536539781373, "kl": 0.022585496503438662, "context": 0.47782928016385995}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.4935726523399353, "loss_scale": 4.0, "retries": 0, "seconds": 270.1, "max_memory_gb": 17.11, "ce": 0.02305912130395882, "kl": 0.073907919046178, "context": 0.06001307908445597}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.05561598017811775, "loss_scale": 4.0, "retries": 0, "seconds": 271.5, "max_memory_gb": 17.11, "ce": 0.003771433315705508, "kl": 0.0015412056836794363, "context": 0.004542036127531901}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 1.3689521551132202, "loss_scale": 4.0, "retries": 0, "seconds": 272.7, "max_memory_gb": 17.11, "ce": 0.16461826662998646, "kl": 0.1824268475174904, "context": 0.2558316979557276}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 1.1837009191513062, "loss_scale": 4.0, "retries": 0, "seconds": 273.9, "max_memory_gb": 17.11, "ce": 0.014473361370619386, "kl": 0.22345624413355836, "context": 0.1262014852836728}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 0.23535016179084778, "loss_scale": 4.0, "retries": 0, "seconds": 275.2, "max_memory_gb": 17.11, "ce": 0.019260001892689615, "kl": 0.0015177993973338744, "context": 0.02001890237443149}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.6696265339851379, "loss_scale": 4.0, "retries": 0, "seconds": 276.3, "max_memory_gb": 17.11, "ce": 0.00395558460149914, "kl": 0.11945975248818286, "context": 0.06368546065641567}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.8021376132965088, "loss_scale": 4.0, "retries": 0, "seconds": 277.6, "max_memory_gb": 17.11, "ce": 0.059978065110044554, "kl": 0.05030880257800163, "context": 0.08513246694928966}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 2.7816965579986572, "loss_scale": 4.0, "retries": 0, "seconds": 278.9, "max_memory_gb": 17.11, "ce": 0.10827672923915088, "kl": 0.011685807761750766, "context": 0.11411963103455491}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 1.5039178133010864, "loss_scale": 4.0, "retries": 0, "seconds": 280.1, "max_memory_gb": 17.11, "ce": 0.043416059750597924, "kl": 0.07870533615641762, "context": 0.08276872773421928}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.16679784655570984, "loss_scale": 4.0, "retries": 0, "seconds": 281.5, "max_memory_gb": 17.11, "ce": 0.006672812509350479, "kl": 0.01594994427432539, "context": 0.014647784671979025}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.7476096749305725, "loss_scale": 4.0, "retries": 0, "seconds": 282.7, "max_memory_gb": 17.11, "ce": 0.030778426415054128, "kl": 0.010130772955562861, "context": 0.03584381201653741}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.33459562063217163, "loss_scale": 4.0, "retries": 0, "seconds": 283.9, "max_memory_gb": 17.11, "ce": 0.013309744783327915, "kl": 0.003327314459511399, "context": 0.01497340208152309}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.31765642762184143, "loss_scale": 4.0, "retries": 0, "seconds": 285.2, "max_memory_gb": 17.11, "ce": 0.011420286624343134, "kl": 0.04162030736915767, "context": 0.03223044006153941}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 1.1085759401321411, "loss_scale": 4.0, "retries": 0, "seconds": 286.4, "max_memory_gb": 17.11, "ce": 0.08025212105712853, "kl": 0.014369776703915704, "context": 0.08743700812919997}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 3.3892228603363037, "loss_scale": 4.0, "retries": 0, "seconds": 287.6, "max_memory_gb": 17.11, "ce": 0.361335045134183, "kl": 0.294317704916466, "context": 0.5084938959917054}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.5741943120956421, "loss_scale": 4.0, "retries": 0, "seconds": 288.9, "max_memory_gb": 17.11, "ce": 0.039435794489691034, "kl": 0.09698597592796432, "context": 0.08792878361418843}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.385498583316803, "loss_scale": 4.0, "retries": 0, "seconds": 290.1, "max_memory_gb": 17.11, "ce": 0.03166045734542422, "kl": 0.003457929731666809, "context": 0.03338942280970514}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.9979995489120483, "loss_scale": 4.0, "retries": 0, "seconds": 291.4, "max_memory_gb": 17.11, "ce": 0.03234526002779603, "kl": 0.17011676053516567, "context": 0.11740363808348775}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.7598906755447388, "loss_scale": 4.0, "retries": 0, "seconds": 292.5, "max_memory_gb": 17.11, "ce": 0.024623000383144245, "kl": 0.02142126305261627, "context": 0.035333632898982614}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 2.4930579662323, "loss_scale": 4.0, "retries": 0, "seconds": 293.8, "max_memory_gb": 17.11, "ce": 0.23993403726490214, "kl": 0.02815279442256724, "context": 0.25401044014142826}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.22578148543834686, "loss_scale": 4.0, "retries": 0, "seconds": 295.1, "max_memory_gb": 17.11, "ce": 0.012171310328994878, "kl": 0.025362566695548594, "context": 0.024852594011463225}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 1.6802659034729004, "loss_scale": 4.0, "retries": 0, "seconds": 296.3, "max_memory_gb": 17.11, "ce": 0.09215709479758516, "kl": 0.015489260399590421, "context": 0.09990172120160423}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.559952437877655, "loss_scale": 4.0, "retries": 0, "seconds": 297.4, "max_memory_gb": 17.11, "ce": 0.024645416808198206, "kl": 0.010893931063037599, "context": 0.03009238332742825}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 1.003128170967102, "loss_scale": 4.0, "retries": 0, "seconds": 298.7, "max_memory_gb": 17.11, "ce": 0.12306190899107605, "kl": 0.034434984117979184, "context": 0.14027940470259637}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 0.31546688079833984, "loss_scale": 4.0, "retries": 0, "seconds": 299.9, "max_memory_gb": 17.11, "ce": 0.033245248429011554, "kl": 0.027375889592804015, "context": 0.04693319252692163}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.59892338514328, "loss_scale": 4.0, "retries": 0, "seconds": 301.1, "max_memory_gb": 17.11, "ce": 0.058357307920232415, "kl": 0.018251716406666674, "context": 0.06748316553421319}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.15558010339736938, "loss_scale": 4.0, "retries": 0, "seconds": 302.4, "max_memory_gb": 17.11, "ce": 0.009628531042835675, "kl": 0.01625888289163413, "context": 0.017757972571416758}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.07871297746896744, "loss_scale": 4.0, "retries": 0, "seconds": 303.6, "max_memory_gb": 17.11, "ce": 0.005249574925983325, "kl": 0.0019273933794465847, "context": 0.00621327172848396}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 2.614694118499756, "loss_scale": 4.0, "retries": 0, "seconds": 304.9, "max_memory_gb": 17.11, "ce": 0.0626090116929845, "kl": 0.18095070192430285, "context": 0.15308436265331693}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 1.9522367715835571, "loss_scale": 4.0, "retries": 0, "seconds": 306.1, "max_memory_gb": 17.11, "ce": 0.3197383629158139, "kl": 0.023706122301518917, "context": 0.3315914256963879}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.024391965940594673, "loss_scale": 4.0, "retries": 0, "seconds": 307.4, "max_memory_gb": 17.11, "ce": 0.0015629102708771825, "kl": 0.00044155685918667587, "context": 0.0017836887418525293}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.9366647601127625, "loss_scale": 4.0, "retries": 0, "seconds": 308.7, "max_memory_gb": 17.11, "ce": 0.011354105692589656, "kl": 0.12319979991843866, "context": 0.07295400701696053}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.31654682755470276, "loss_scale": 4.0, "retries": 0, "seconds": 309.9, "max_memory_gb": 17.11, "ce": 0.0215646312572062, "kl": 0.011111961444839835, "context": 0.027120611630380154}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.9974693655967712, "loss_scale": 4.0, "retries": 0, "seconds": 311.2, "max_memory_gb": 17.11, "ce": 0.09020777186378837, "kl": 0.08263261790852994, "context": 0.13152407761663198}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.154109388589859, "loss_scale": 4.0, "retries": 0, "seconds": 312.5, "max_memory_gb": 17.11, "ce": 0.009351978384074755, "kl": 0.004550708290480543, "context": 0.011627332482021302}

runs/fixed_internvl/context/Middle8_s42/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Middle8_s42/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.14.mlp.down_proj": 144,
+    "model.language_model.layers.14.mlp.gate_proj": 144,
+    "model.language_model.layers.14.mlp.up_proj": 144,
+    "model.language_model.layers.14.self_attn.k_proj": 144,
+    "model.language_model.layers.14.self_attn.o_proj": 144,
+    "model.language_model.layers.14.self_attn.q_proj": 144,
+    "model.language_model.layers.14.self_attn.v_proj": 144,
+    "model.language_model.layers.15.mlp.down_proj": 144,
+    "model.language_model.layers.15.mlp.gate_proj": 144,
+    "model.language_model.layers.15.mlp.up_proj": 144,
+    "model.language_model.layers.15.self_attn.k_proj": 144,
+    "model.language_model.layers.15.self_attn.o_proj": 144,
+    "model.language_model.layers.15.self_attn.q_proj": 144,
+    "model.language_model.layers.15.self_attn.v_proj": 144,
+    "model.language_model.layers.16.mlp.down_proj": 144,
+    "model.language_model.layers.16.mlp.gate_proj": 144,
+    "model.language_model.layers.16.mlp.up_proj": 144,
+    "model.language_model.layers.16.self_attn.k_proj": 144,
+    "model.language_model.layers.16.self_attn.o_proj": 144,
+    "model.language_model.layers.16.self_attn.q_proj": 144,
+    "model.language_model.layers.16.self_attn.v_proj": 144,
+    "model.language_model.layers.17.mlp.down_proj": 144,
+    "model.language_model.layers.17.mlp.gate_proj": 144,
+    "model.language_model.layers.17.mlp.up_proj": 144,
+    "model.language_model.layers.17.self_attn.k_proj": 144,
+    "model.language_model.layers.17.self_attn.o_proj": 144,
+    "model.language_model.layers.17.self_attn.q_proj": 144,
+    "model.language_model.layers.17.self_attn.v_proj": 144,
+    "model.language_model.layers.18.mlp.down_proj": 144,
+    "model.language_model.layers.18.mlp.gate_proj": 144,
+    "model.language_model.layers.18.mlp.up_proj": 144,
+    "model.language_model.layers.18.self_attn.k_proj": 144,
+    "model.language_model.layers.18.self_attn.o_proj": 144,
+    "model.language_model.layers.18.self_attn.q_proj": 144,
+    "model.language_model.layers.18.self_attn.v_proj": 144,
+    "model.language_model.layers.19.mlp.down_proj": 144,
+    "model.language_model.layers.19.mlp.gate_proj": 144,
+    "model.language_model.layers.19.mlp.up_proj": 144,
+    "model.language_model.layers.19.self_attn.k_proj": 144,
+    "model.language_model.layers.19.self_attn.o_proj": 144,
+    "model.language_model.layers.19.self_attn.q_proj": 144,
+    "model.language_model.layers.19.self_attn.v_proj": 144,
+    "model.language_model.layers.20.mlp.down_proj": 144,
+    "model.language_model.layers.20.mlp.gate_proj": 144,
+    "model.language_model.layers.20.mlp.up_proj": 144,
+    "model.language_model.layers.20.self_attn.k_proj": 144,
+    "model.language_model.layers.20.self_attn.o_proj": 144,
+    "model.language_model.layers.20.self_attn.q_proj": 144,
+    "model.language_model.layers.20.self_attn.v_proj": 144,
+    "model.language_model.layers.21.mlp.down_proj": 144,
+    "model.language_model.layers.21.mlp.gate_proj": 144,
+    "model.language_model.layers.21.mlp.up_proj": 144,
+    "model.language_model.layers.21.self_attn.k_proj": 144,
+    "model.language_model.layers.21.self_attn.o_proj": 144,
+    "model.language_model.layers.21.self_attn.q_proj": 144,
+    "model.language_model.layers.21.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.14.mlp.down_proj": 72,
+    "model.language_model.layers.14.mlp.gate_proj": 72,
+    "model.language_model.layers.14.mlp.up_proj": 72,
+    "model.language_model.layers.14.self_attn.k_proj": 72,
+    "model.language_model.layers.14.self_attn.o_proj": 72,
+    "model.language_model.layers.14.self_attn.q_proj": 72,
+    "model.language_model.layers.14.self_attn.v_proj": 72,
+    "model.language_model.layers.15.mlp.down_proj": 72,
+    "model.language_model.layers.15.mlp.gate_proj": 72,
+    "model.language_model.layers.15.mlp.up_proj": 72,
+    "model.language_model.layers.15.self_attn.k_proj": 72,
+    "model.language_model.layers.15.self_attn.o_proj": 72,
+    "model.language_model.layers.15.self_attn.q_proj": 72,
+    "model.language_model.layers.15.self_attn.v_proj": 72,
+    "model.language_model.layers.16.mlp.down_proj": 72,
+    "model.language_model.layers.16.mlp.gate_proj": 72,
+    "model.language_model.layers.16.mlp.up_proj": 72,
+    "model.language_model.layers.16.self_attn.k_proj": 72,
+    "model.language_model.layers.16.self_attn.o_proj": 72,
+    "model.language_model.layers.16.self_attn.q_proj": 72,
+    "model.language_model.layers.16.self_attn.v_proj": 72,
+    "model.language_model.layers.17.mlp.down_proj": 72,
+    "model.language_model.layers.17.mlp.gate_proj": 72,
+    "model.language_model.layers.17.mlp.up_proj": 72,
+    "model.language_model.layers.17.self_attn.k_proj": 72,
+    "model.language_model.layers.17.self_attn.o_proj": 72,
+    "model.language_model.layers.17.self_attn.q_proj": 72,
+    "model.language_model.layers.17.self_attn.v_proj": 72,
+    "model.language_model.layers.18.mlp.down_proj": 72,
+    "model.language_model.layers.18.mlp.gate_proj": 72,
+    "model.language_model.layers.18.mlp.up_proj": 72,
+    "model.language_model.layers.18.self_attn.k_proj": 72,
+    "model.language_model.layers.18.self_attn.o_proj": 72,
+    "model.language_model.layers.18.self_attn.q_proj": 72,
+    "model.language_model.layers.18.self_attn.v_proj": 72,
+    "model.language_model.layers.19.mlp.down_proj": 72,
+    "model.language_model.layers.19.mlp.gate_proj": 72,
+    "model.language_model.layers.19.mlp.up_proj": 72,
+    "model.language_model.layers.19.self_attn.k_proj": 72,
+    "model.language_model.layers.19.self_attn.o_proj": 72,
+    "model.language_model.layers.19.self_attn.q_proj": 72,
+    "model.language_model.layers.19.self_attn.v_proj": 72,
+    "model.language_model.layers.20.mlp.down_proj": 72,
+    "model.language_model.layers.20.mlp.gate_proj": 72,
+    "model.language_model.layers.20.mlp.up_proj": 72,
+    "model.language_model.layers.20.self_attn.k_proj": 72,
+    "model.language_model.layers.20.self_attn.o_proj": 72,
+    "model.language_model.layers.20.self_attn.q_proj": 72,
+    "model.language_model.layers.20.self_attn.v_proj": 72,
+    "model.language_model.layers.21.mlp.down_proj": 72,
+    "model.language_model.layers.21.mlp.gate_proj": 72,
+    "model.language_model.layers.21.mlp.up_proj": 72,
+    "model.language_model.layers.21.self_attn.k_proj": 72,
+    "model.language_model.layers.21.self_attn.o_proj": 72,
+    "model.language_model.layers.21.self_attn.q_proj": 72,
+    "model.language_model.layers.21.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.14.self_attn.k_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.14.self_attn.v_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.14.mlp.gate_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.20.self_attn.v_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.14.mlp.down_proj",
+    "model.language_model.layers.14.mlp.up_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.k_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.14.self_attn.o_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.17.self_attn.v_proj",
+    "model.language_model.layers.14.self_attn.q_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.17.self_attn.o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Middle8_s42/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 42,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Middle8",
+    "layers": [
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21
+    ],
+    "ranks": {
+      "14": 72,
+      "15": 72,
+      "16": 72,
+      "17": 72,
+      "18": 72,
+      "19": 72,
+      "20": 72,
+      "21": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Middle8_s42/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 16.130489349365234, "loss_scale": 4.0, "retries": 0, "seconds": 2.2, "max_memory_gb": 16.79, "ce": 0.6558216065168381, "kl": 0.11560958076734096, "context": 0.7136263996362686}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 9.370607376098633, "loss_scale": 4.0, "retries": 0, "seconds": 3.5, "max_memory_gb": 17.1, "ce": 0.5915933772915025, "kl": 0.05973740482295398, "context": 0.6214620678285883}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 22.091083526611328, "loss_scale": 4.0, "retries": 0, "seconds": 4.7, "max_memory_gb": 17.1, "ce": 0.5744987092912197, "kl": 0.0313060994958505, "context": 0.5901517658494413}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 14.230883598327637, "loss_scale": 4.0, "retries": 0, "seconds": 5.9, "max_memory_gb": 17.1, "ce": 0.2709583453834057, "kl": 0.6745437646750361, "context": 0.6082302253926173}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 5.2085981369018555, "loss_scale": 4.0, "retries": 0, "seconds": 7.2, "max_memory_gb": 17.1, "ce": 0.16661406587809324, "kl": 1.1540975960961077, "context": 0.7436628434224986}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 9.823942184448242, "loss_scale": 4.0, "retries": 0, "seconds": 8.3, "max_memory_gb": 17.1, "ce": 0.11135768459644169, "kl": 0.31877818671637215, "context": 0.27074677823111415}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 1.4812182188034058, "loss_scale": 4.0, "retries": 0, "seconds": 9.5, "max_memory_gb": 17.1, "ce": 0.088130101445131, "kl": 0.36709878634428605, "context": 0.27167949359863997}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 3.2901670932769775, "loss_scale": 4.0, "retries": 0, "seconds": 10.8, "max_memory_gb": 17.1, "ce": 0.2249841856587409, "kl": 0.0853788318338502, "context": 0.26767359971290716}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 6.761363983154297, "loss_scale": 4.0, "retries": 0, "seconds": 12.1, "max_memory_gb": 17.1, "ce": 0.5051301949715707, "kl": 0.16652349673677236, "context": 0.5883919633924961}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 5.0244011878967285, "loss_scale": 4.0, "retries": 0, "seconds": 13.3, "max_memory_gb": 17.1, "ce": 0.033976752805756405, "kl": 0.25386147294193506, "context": 0.16090749180875719}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.600631833076477, "loss_scale": 4.0, "retries": 0, "seconds": 14.5, "max_memory_gb": 17.1, "ce": 0.001963089698620024, "kl": 0.23746182624574885, "context": 0.1206940043321083}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 9.704194068908691, "loss_scale": 4.0, "retries": 0, "seconds": 15.7, "max_memory_gb": 17.11, "ce": 0.3625696170493029, "kl": 1.0022738781990483, "context": 0.8637065298389643}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.8638846278190613, "loss_scale": 4.0, "retries": 0, "seconds": 16.9, "max_memory_gb": 17.11, "ce": 0.06097183306701481, "kl": 0.04848133632913232, "context": 0.08521249995101243}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 0.5719956755638123, "loss_scale": 4.0, "retries": 0, "seconds": 17.8, "max_memory_gb": 17.11, "ce": 0.024656656954903156, "kl": 0.10407631250564009, "context": 0.07669481518678367}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 6.215385913848877, "loss_scale": 4.0, "retries": 0, "seconds": 18.8, "max_memory_gb": 17.11, "ce": 0.2912162165157497, "kl": 0.07195328804664314, "context": 0.32719285786151886}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 1.4374216794967651, "loss_scale": 4.0, "retries": 0, "seconds": 20.0, "max_memory_gb": 17.11, "ce": 0.18261839163324112, "kl": 0.020212214176581256, "context": 0.19272450512426076}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 0.5236051678657532, "loss_scale": 4.0, "retries": 0, "seconds": 20.9, "max_memory_gb": 17.11, "ce": 0.0054562786826863885, "kl": 0.08317770012217807, "context": 0.047045128798345104}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 1.4452226161956787, "loss_scale": 4.0, "retries": 0, "seconds": 21.9, "max_memory_gb": 17.11, "ce": 0.07578444859245792, "kl": 0.0406863912648987, "context": 0.09612764790654182}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.2415304332971573, "loss_scale": 4.0, "retries": 0, "seconds": 23.0, "max_memory_gb": 17.11, "ce": 0.004243847440193349, "kl": 0.01809718373715441, "context": 0.013292439407450729}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.40940266847610474, "loss_scale": 4.0, "retries": 0, "seconds": 24.1, "max_memory_gb": 17.11, "ce": 0.004155492417339701, "kl": 0.08887541803687782, "context": 0.04859320328978356}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.33675000071525574, "loss_scale": 4.0, "retries": 0, "seconds": 25.3, "max_memory_gb": 17.11, "ce": 0.004976553638698533, "kl": 0.022554861317985342, "context": 0.01625398371834308}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.3279649615287781, "loss_scale": 4.0, "retries": 0, "seconds": 26.5, "max_memory_gb": 17.11, "ce": 0.018314073440706125, "kl": 0.008895132061297772, "context": 0.022761638918382232}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 3.8103344440460205, "loss_scale": 4.0, "retries": 0, "seconds": 27.7, "max_memory_gb": 17.11, "ce": 0.06598842114908621, "kl": 0.35077803080503145, "context": 0.24137743475148454}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 1.8994321823120117, "loss_scale": 4.0, "retries": 0, "seconds": 28.8, "max_memory_gb": 17.11, "ce": 0.05269682796642883, "kl": 0.7491692899420741, "context": 0.42728148102469277}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 1.2933595180511475, "loss_scale": 4.0, "retries": 0, "seconds": 29.9, "max_memory_gb": 17.11, "ce": 0.10313957316975575, "kl": 0.17577181597880553, "context": 0.1910254811809864}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 2.7640464305877686, "loss_scale": 4.0, "retries": 0, "seconds": 31.0, "max_memory_gb": 17.11, "ce": 0.07522123892613308, "kl": 0.00045145940509883076, "context": 0.07544697209050355}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 12.963387489318848, "loss_scale": 4.0, "retries": 0, "seconds": 32.4, "max_memory_gb": 17.11, "ce": 0.9023800544746337, "kl": 0.014271825446485309, "context": 0.9095159955177223}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 2.1570355892181396, "loss_scale": 4.0, "retries": 0, "seconds": 33.7, "max_memory_gb": 17.11, "ce": 0.0069295946741476655, "kl": 0.8273716660332866, "context": 0.42061542510055006}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 1.66593337059021, "loss_scale": 4.0, "retries": 0, "seconds": 34.9, "max_memory_gb": 17.11, "ce": 0.12952983163995668, "kl": 0.025482794153504074, "context": 0.14227122836746275}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 1.7948857545852661, "loss_scale": 4.0, "retries": 0, "seconds": 36.3, "max_memory_gb": 17.11, "ce": 0.01478132518241182, "kl": 0.5495447283319663, "context": 0.2895536944270134}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 3.558422803878784, "loss_scale": 4.0, "retries": 0, "seconds": 37.3, "max_memory_gb": 17.11, "ce": 0.5268092317855917, "kl": 0.43637307587778196, "context": 0.7449957458302379}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 1.1822503805160522, "loss_scale": 4.0, "retries": 0, "seconds": 38.6, "max_memory_gb": 17.11, "ce": 0.05017163907177746, "kl": 0.027718621218809858, "context": 0.06403094960842282}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.04567371681332588, "loss_scale": 4.0, "retries": 0, "seconds": 40.0, "max_memory_gb": 17.11, "ce": 0.0020366765129438136, "kl": 0.0008302033093059435, "context": 0.002451778098475188}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.11395701766014099, "loss_scale": 4.0, "retries": 0, "seconds": 41.2, "max_memory_gb": 17.11, "ce": 0.004380637023132294, "kl": 0.0002514159114070935, "context": 0.004506345052504912}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 1.402202844619751, "loss_scale": 4.0, "retries": 0, "seconds": 42.6, "max_memory_gb": 17.11, "ce": 0.08596163941547275, "kl": 0.033342617032758426, "context": 0.10263294691685587}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 1.516953945159912, "loss_scale": 4.0, "retries": 0, "seconds": 43.8, "max_memory_gb": 17.11, "ce": 0.14326716577852494, "kl": 0.02380619753694191, "context": 0.1551702673350519}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 0.7961521744728088, "loss_scale": 4.0, "retries": 0, "seconds": 45.0, "max_memory_gb": 17.11, "ce": 0.0701188578386791, "kl": 0.3130544191226363, "context": 0.22664607467595488}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.8520469069480896, "loss_scale": 4.0, "retries": 0, "seconds": 46.3, "max_memory_gb": 17.11, "ce": 0.026421296875923872, "kl": 0.07191377342678607, "context": 0.06237818393856287}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.9164760112762451, "loss_scale": 4.0, "retries": 0, "seconds": 47.5, "max_memory_gb": 17.11, "ce": 0.24151015793904662, "kl": 0.18296208957326598, "context": 0.33299120189622045}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.4582378566265106, "loss_scale": 4.0, "retries": 0, "seconds": 48.7, "max_memory_gb": 17.11, "ce": 0.021759125927928835, "kl": 0.006566796406332287, "context": 0.02504252414655639}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 0.4795399010181427, "loss_scale": 4.0, "retries": 0, "seconds": 50.0, "max_memory_gb": 17.11, "ce": 0.02030857036152156, "kl": 0.03373774670762941, "context": 0.037177444086410105}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 1.5120264291763306, "loss_scale": 4.0, "retries": 0, "seconds": 51.2, "max_memory_gb": 17.11, "ce": 0.013474777981173247, "kl": 0.5785139928921126, "context": 0.3027317743981257}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 7.599041938781738, "loss_scale": 4.0, "retries": 0, "seconds": 52.4, "max_memory_gb": 17.11, "ce": 0.2464936342148576, "kl": 0.2971533149411698, "context": 0.39507029126252746}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 5.07863712310791, "loss_scale": 4.0, "retries": 0, "seconds": 53.6, "max_memory_gb": 17.11, "ce": 0.24462272156961262, "kl": 0.04057742148870602, "context": 0.26491143996827304}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 6.878861904144287, "loss_scale": 4.0, "retries": 0, "seconds": 54.9, "max_memory_gb": 17.11, "ce": 0.20671999373007566, "kl": 0.056280991637322586, "context": 0.23486048955237493}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 1.3380205631256104, "loss_scale": 4.0, "retries": 0, "seconds": 56.2, "max_memory_gb": 17.11, "ce": 0.21730980079155415, "kl": 0.049627126652922016, "context": 0.24212335655465722}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.3671227693557739, "loss_scale": 4.0, "retries": 0, "seconds": 57.4, "max_memory_gb": 17.11, "ce": 0.017270541982725263, "kl": 0.013460539892548695, "context": 0.024000811856240034}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.8681777119636536, "loss_scale": 4.0, "retries": 0, "seconds": 58.6, "max_memory_gb": 17.11, "ce": 0.09018924784322735, "kl": 0.028571277399294104, "context": 0.10447488726640586}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 2.7666709423065186, "loss_scale": 4.0, "retries": 0, "seconds": 59.9, "max_memory_gb": 17.11, "ce": 0.2522659435089736, "kl": 0.09017121375472925, "context": 0.2973515579396917}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.6063137054443359, "loss_scale": 4.0, "retries": 0, "seconds": 61.1, "max_memory_gb": 17.11, "ce": 0.015855005010962486, "kl": 0.08079202007502317, "context": 0.05625101667828858}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 2.1930105686187744, "loss_scale": 4.0, "retries": 0, "seconds": 62.3, "max_memory_gb": 17.11, "ce": 0.04525822459254414, "kl": 0.14089598583814222, "context": 0.11570622038561851}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.2940649688243866, "loss_scale": 4.0, "retries": 0, "seconds": 63.6, "max_memory_gb": 17.11, "ce": 0.011340125747665297, "kl": 0.008322382635753911, "context": 0.015501317066082265}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 0.33309438824653625, "loss_scale": 4.0, "retries": 0, "seconds": 64.8, "max_memory_gb": 17.11, "ce": 0.005072854779427871, "kl": 0.023277610082004685, "context": 0.01671166007872671}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 3.280212163925171, "loss_scale": 4.0, "retries": 0, "seconds": 66.0, "max_memory_gb": 17.11, "ce": 0.07270095478816074, "kl": 0.07300504553495557, "context": 0.10920347941282671}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 13.839298248291016, "loss_scale": 4.0, "retries": 0, "seconds": 67.3, "max_memory_gb": 17.11, "ce": 0.7565817023787531, "kl": 0.5411809067936701, "context": 1.0271721217832237}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.11870769411325455, "loss_scale": 4.0, "retries": 0, "seconds": 68.5, "max_memory_gb": 17.11, "ce": 0.004821976515813731, "kl": 0.0021916546397733327, "context": 0.005917803660850041}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 2.0793213844299316, "loss_scale": 4.0, "retries": 0, "seconds": 69.6, "max_memory_gb": 17.11, "ce": 0.05135238332695735, "kl": 0.01019272280518635, "context": 0.05644874464269378}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 1.4590966701507568, "loss_scale": 4.0, "retries": 0, "seconds": 70.9, "max_memory_gb": 17.11, "ce": 0.015319328216719441, "kl": 0.33112011107368744, "context": 0.1808793797972612}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.16489079594612122, "loss_scale": 4.0, "retries": 0, "seconds": 72.1, "max_memory_gb": 17.11, "ce": 0.007777969383823802, "kl": 0.000424940789230277, "context": 0.007990440033609048}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 5.381213665008545, "loss_scale": 4.0, "retries": 0, "seconds": 73.4, "max_memory_gb": 17.11, "ce": 0.11502492652834917, "kl": 0.2096106211829465, "context": 0.21983024409746577}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 0.6887214183807373, "loss_scale": 4.0, "retries": 0, "seconds": 74.6, "max_memory_gb": 17.11, "ce": 0.005336250542313792, "kl": 0.13661879420396872, "context": 0.07364564822637476}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.6399410367012024, "loss_scale": 4.0, "retries": 0, "seconds": 75.8, "max_memory_gb": 17.11, "ce": 0.0025120220707322005, "kl": 0.12021268756689096, "context": 0.06261836537305498}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 3.0678470134735107, "loss_scale": 4.0, "retries": 0, "seconds": 77.1, "max_memory_gb": 17.11, "ce": 0.17885791153821629, "kl": 0.16209928125257278, "context": 0.25990755379461916}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.7827238440513611, "loss_scale": 4.0, "retries": 0, "seconds": 78.3, "max_memory_gb": 17.11, "ce": 0.03413836044637719, "kl": 0.048023262239439646, "context": 0.058149989730736706}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 7.658172130584717, "loss_scale": 4.0, "retries": 0, "seconds": 79.5, "max_memory_gb": 17.11, "ce": 0.18413625104585662, "kl": 0.07534570843563415, "context": 0.22180909756571054}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 15.716806411743164, "loss_scale": 4.0, "retries": 0, "seconds": 80.8, "max_memory_gb": 17.11, "ce": 0.16764811102530075, "kl": 0.17422558932275933, "context": 0.2547608907798349}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.22389812767505646, "loss_scale": 4.0, "retries": 0, "seconds": 82.0, "max_memory_gb": 17.11, "ce": 0.012628072385268752, "kl": 0.0029035371549070987, "context": 0.01407984097750159}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.693892240524292, "loss_scale": 4.0, "retries": 0, "seconds": 83.4, "max_memory_gb": 17.11, "ce": 0.0287919792572211, "kl": 0.023488222039304674, "context": 0.040536091379181016}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 1.7196226119995117, "loss_scale": 4.0, "retries": 0, "seconds": 84.5, "max_memory_gb": 17.11, "ce": 0.13113595748473017, "kl": 0.049042222133721225, "context": 0.15565707531641237}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 4.325706481933594, "loss_scale": 4.0, "retries": 0, "seconds": 85.7, "max_memory_gb": 17.11, "ce": 0.3666863873731927, "kl": 0.14658041229631635, "context": 0.4399765935522737}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 6.3669514656066895, "loss_scale": 4.0, "retries": 0, "seconds": 87.1, "max_memory_gb": 17.11, "ce": 0.671173560549505, "kl": 0.045624818943906575, "context": 0.6939859476406127}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.0681176707148552, "loss_scale": 4.0, "retries": 0, "seconds": 88.3, "max_memory_gb": 17.11, "ce": 0.00239723028971639, "kl": 0.0018829573855327908, "context": 0.003338708949740976}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 4.864058971405029, "loss_scale": 4.0, "retries": 0, "seconds": 89.5, "max_memory_gb": 17.11, "ce": 0.4135823680990143, "kl": 0.01792314020349295, "context": 0.4225439484434901}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 2.5131561756134033, "loss_scale": 4.0, "retries": 0, "seconds": 90.8, "max_memory_gb": 17.11, "ce": 0.0039518528210464865, "kl": 0.12897590804641368, "context": 0.06843980646226555}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.054248571395874, "loss_scale": 4.0, "retries": 0, "seconds": 91.9, "max_memory_gb": 17.11, "ce": 0.015891210052359384, "kl": 0.15577458426014346, "context": 0.09377850008604582}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 1.7554576396942139, "loss_scale": 4.0, "retries": 0, "seconds": 92.9, "max_memory_gb": 17.11, "ce": 0.0072245958872372285, "kl": 0.29650716530159116, "context": 0.15547817456535995}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 1.3098818063735962, "loss_scale": 4.0, "retries": 0, "seconds": 94.0, "max_memory_gb": 17.11, "ce": 0.0023407309454341885, "kl": 0.24096464071840273, "context": 0.12282304786640452}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.6872392296791077, "loss_scale": 4.0, "retries": 0, "seconds": 95.0, "max_memory_gb": 17.11, "ce": 0.007061184282065369, "kl": 0.10641948129341472, "context": 0.06027092321892269}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 1.2599042654037476, "loss_scale": 4.0, "retries": 0, "seconds": 96.3, "max_memory_gb": 17.11, "ce": 0.014333117935166229, "kl": 0.3458381257023575, "context": 0.1872521863770089}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 1.546478509902954, "loss_scale": 4.0, "retries": 0, "seconds": 97.5, "max_memory_gb": 17.11, "ce": 0.18579172881072736, "kl": 0.05211275973283591, "context": 0.21184811242892465}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 4.4381022453308105, "loss_scale": 4.0, "retries": 0, "seconds": 98.7, "max_memory_gb": 17.11, "ce": 0.7142151395237306, "kl": 0.06408793509535826, "context": 0.7462591331277508}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 1.0611494779586792, "loss_scale": 4.0, "retries": 0, "seconds": 100.0, "max_memory_gb": 17.11, "ce": 0.0008068629758781753, "kl": 0.15111043702881943, "context": 0.07636208354961127}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 0.8185192942619324, "loss_scale": 4.0, "retries": 0, "seconds": 101.3, "max_memory_gb": 17.11, "ce": 0.06801588789676316, "kl": 0.06547366754966788, "context": 0.1007527211913839}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 0.8293208479881287, "loss_scale": 4.0, "retries": 0, "seconds": 102.5, "max_memory_gb": 17.11, "ce": 0.04943283063767012, "kl": 0.15743434708565474, "context": 0.12815000675618649}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 2.40036678314209, "loss_scale": 4.0, "retries": 0, "seconds": 103.7, "max_memory_gb": 17.11, "ce": 0.09450120896508452, "kl": 0.08319364284966468, "context": 0.13609803411600296}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 0.6040483117103577, "loss_scale": 4.0, "retries": 0, "seconds": 104.7, "max_memory_gb": 17.11, "ce": 0.017087766143959016, "kl": 0.016774775496742222, "context": 0.025475154194282368}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.9035158753395081, "loss_scale": 4.0, "retries": 0, "seconds": 105.9, "max_memory_gb": 17.11, "ce": 0.08415390579830273, "kl": 0.03346372787245855, "context": 0.10088577345595695}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.7370001077651978, "loss_scale": 4.0, "retries": 0, "seconds": 107.2, "max_memory_gb": 17.11, "ce": 0.028477245119574945, "kl": 0.0048708746617194265, "context": 0.03091268338903319}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.18351644277572632, "loss_scale": 4.0, "retries": 0, "seconds": 108.4, "max_memory_gb": 17.11, "ce": 0.006944867993297521, "kl": 0.020410515062394552, "context": 0.017150125873740762}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.05190332978963852, "loss_scale": 4.0, "retries": 0, "seconds": 109.5, "max_memory_gb": 17.11, "ce": 0.0017425888872821815, "kl": 0.006144518607470673, "context": 0.0048148482528631575}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 0.34829309582710266, "loss_scale": 4.0, "retries": 0, "seconds": 110.9, "max_memory_gb": 17.11, "ce": 0.013629732489789603, "kl": 0.026138077198993415, "context": 0.026698771864175797}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 0.5737846493721008, "loss_scale": 4.0, "retries": 0, "seconds": 112.1, "max_memory_gb": 17.11, "ce": 0.05737946880981326, "kl": 0.022850633540656418, "context": 0.06880478654056787}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 4.134633541107178, "loss_scale": 4.0, "retries": 0, "seconds": 113.4, "max_memory_gb": 17.11, "ce": 0.1808545199164655, "kl": 0.400943974222173, "context": 0.3813265047792811}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 2.823063611984253, "loss_scale": 4.0, "retries": 0, "seconds": 114.6, "max_memory_gb": 17.11, "ce": 0.1430898443722981, "kl": 0.48986756172962487, "context": 0.3880236418917775}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 0.9357431530952454, "loss_scale": 4.0, "retries": 0, "seconds": 115.8, "max_memory_gb": 17.11, "ce": 0.03706355392932892, "kl": 0.08865175396203995, "context": 0.08138942904770374}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 0.8378850221633911, "loss_scale": 4.0, "retries": 0, "seconds": 117.1, "max_memory_gb": 17.11, "ce": 0.009786889888346195, "kl": 0.30434528361365665, "context": 0.16195953194983304}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.24929821491241455, "loss_scale": 4.0, "retries": 0, "seconds": 118.3, "max_memory_gb": 17.11, "ce": 0.003033688699360937, "kl": 0.035436834521533456, "context": 0.020752106909640133}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.6615172028541565, "loss_scale": 4.0, "retries": 0, "seconds": 119.5, "max_memory_gb": 17.11, "ce": 0.029368507286562817, "kl": 0.03686437782357643, "context": 0.047800697404454695}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 1.7565947771072388, "loss_scale": 4.0, "retries": 0, "seconds": 120.8, "max_memory_gb": 17.11, "ce": 0.0335707477570395, "kl": 0.07534935911189677, "context": 0.07124542659948929}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.10201473534107208, "loss_scale": 4.0, "retries": 0, "seconds": 122.0, "max_memory_gb": 17.11, "ce": 0.004969131347024813, "kl": 0.0036293168595875613, "context": 0.0067837897659046575}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 2.443024158477783, "loss_scale": 4.0, "retries": 0, "seconds": 123.4, "max_memory_gb": 17.11, "ce": 0.2380112104156069, "kl": 0.029583348639334872, "context": 0.25280288658541394}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.14438724517822266, "loss_scale": 4.0, "retries": 0, "seconds": 124.6, "max_memory_gb": 17.11, "ce": 0.003327533217088785, "kl": 0.01680428825341096, "context": 0.01172967734055419}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 1.6979228258132935, "loss_scale": 4.0, "retries": 0, "seconds": 125.8, "max_memory_gb": 17.11, "ce": 0.18872308963909745, "kl": 0.11433695982123027, "context": 0.24589156860020012}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 1.6323130130767822, "loss_scale": 4.0, "retries": 0, "seconds": 127.1, "max_memory_gb": 17.11, "ce": 0.1484452903678175, "kl": 0.004789733580764732, "context": 0.15084015621687286}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 5.9652886390686035, "loss_scale": 4.0, "retries": 0, "seconds": 128.3, "max_memory_gb": 17.11, "ce": 0.09683529714857286, "kl": 0.33945815869919826, "context": 0.2665643704449394}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 2.148765802383423, "loss_scale": 4.0, "retries": 0, "seconds": 129.5, "max_memory_gb": 17.11, "ce": 0.07467700979032088, "kl": 0.058636548060349014, "context": 0.10399528194830054}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 2.3070638179779053, "loss_scale": 4.0, "retries": 0, "seconds": 130.9, "max_memory_gb": 17.11, "ce": 0.19788322699605487, "kl": 0.12309899930551182, "context": 0.25943272293079644}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 2.7809622287750244, "loss_scale": 4.0, "retries": 0, "seconds": 132.2, "max_memory_gb": 17.11, "ce": 0.23006832618557382, "kl": 0.002385423933446873, "context": 0.23126104280527215}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 1.8821327686309814, "loss_scale": 4.0, "retries": 0, "seconds": 133.4, "max_memory_gb": 17.11, "ce": 0.03324253766913898, "kl": 0.48618983419248707, "context": 0.2763374619826209}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 4.698672771453857, "loss_scale": 4.0, "retries": 0, "seconds": 134.7, "max_memory_gb": 17.11, "ce": 0.12343032174976543, "kl": 0.010582447554043029, "context": 0.12872154643991962}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.02165120095014572, "loss_scale": 4.0, "retries": 0, "seconds": 136.0, "max_memory_gb": 17.11, "ce": 0.00031560879142489284, "kl": 0.001483791634541376, "context": 0.001057504618074745}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 3.3780713081359863, "loss_scale": 4.0, "retries": 0, "seconds": 137.3, "max_memory_gb": 17.11, "ce": 0.09408271189022344, "kl": 0.1997270482388558, "context": 0.1939462285663467}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 2.0319738388061523, "loss_scale": 4.0, "retries": 0, "seconds": 138.5, "max_memory_gb": 17.11, "ce": 0.5575245537474984, "kl": 0.12765039983787574, "context": 0.6213497536955401}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 0.17459933459758759, "loss_scale": 4.0, "retries": 0, "seconds": 139.7, "max_memory_gb": 17.11, "ce": 0.009770559787284583, "kl": 0.002006524216994876, "context": 0.01077382177754771}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 6.336576461791992, "loss_scale": 4.0, "retries": 0, "seconds": 141.0, "max_memory_gb": 17.11, "ce": 0.3107062818307895, "kl": 0.029178516604588367, "context": 0.3252955492644105}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 3.362835168838501, "loss_scale": 4.0, "retries": 0, "seconds": 142.2, "max_memory_gb": 17.11, "ce": 0.3369203094189288, "kl": 0.0014400200379895978, "context": 0.3376403103757184}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 3.8367254734039307, "loss_scale": 4.0, "retries": 0, "seconds": 143.4, "max_memory_gb": 17.11, "ce": 0.5609962409034779, "kl": 0.02635428886515001, "context": 0.5741733890681644}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 2.617022752761841, "loss_scale": 4.0, "retries": 0, "seconds": 144.7, "max_memory_gb": 17.11, "ce": 0.09063427834189497, "kl": 0.9535517122130841, "context": 0.5674101205804618}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.4128236174583435, "loss_scale": 4.0, "retries": 0, "seconds": 145.9, "max_memory_gb": 17.11, "ce": 0.02072241488713189, "kl": 0.00761944909390877, "context": 0.024532139614166226}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 1.4826947450637817, "loss_scale": 4.0, "retries": 0, "seconds": 147.1, "max_memory_gb": 17.11, "ce": 0.02932448260980891, "kl": 0.20956038170646707, "context": 0.1341046714260301}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 1.3233765363693237, "loss_scale": 4.0, "retries": 0, "seconds": 148.4, "max_memory_gb": 17.11, "ce": 0.02163738761737477, "kl": 0.09536071148841074, "context": 0.06931774499389576}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 0.4145926237106323, "loss_scale": 4.0, "retries": 0, "seconds": 149.6, "max_memory_gb": 17.11, "ce": 0.004074953845702112, "kl": 0.06973926321006729, "context": 0.038944585365243256}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.15948665142059326, "loss_scale": 4.0, "retries": 0, "seconds": 150.8, "max_memory_gb": 17.11, "ce": 0.009143611212493852, "kl": 0.0027960148254351225, "context": 0.010541618627030402}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 3.4763717651367188, "loss_scale": 4.0, "retries": 0, "seconds": 152.1, "max_memory_gb": 17.11, "ce": 0.09403064307116438, "kl": 0.666892679844139, "context": 0.42747698300445336}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.7561431527137756, "loss_scale": 4.0, "retries": 0, "seconds": 153.3, "max_memory_gb": 17.11, "ce": 0.008665415924042463, "kl": 0.0635367224458605, "context": 0.04043377804919146}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 3.5781302452087402, "loss_scale": 4.0, "retries": 0, "seconds": 154.6, "max_memory_gb": 17.11, "ce": 0.5737991511705332, "kl": 0.19660972377099029, "context": 0.6721039983676746}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.8656237721443176, "loss_scale": 4.0, "retries": 0, "seconds": 155.8, "max_memory_gb": 17.11, "ce": 0.0024862301943358034, "kl": 0.08329685567878187, "context": 0.044134658557595685}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.9059239625930786, "loss_scale": 4.0, "retries": 0, "seconds": 157.0, "max_memory_gb": 17.11, "ce": 0.03886453856830485, "kl": 0.055632180607062764, "context": 0.06668062931566965}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 5.162151336669922, "loss_scale": 4.0, "retries": 0, "seconds": 158.3, "max_memory_gb": 17.11, "ce": 0.1799718207475962, "kl": 0.13381939241185137, "context": 0.24688151695590932}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 1.983674168586731, "loss_scale": 4.0, "retries": 0, "seconds": 159.5, "max_memory_gb": 17.11, "ce": 0.05092179587518331, "kl": 0.08582582481903955, "context": 0.0938347089686431}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 0.6954466700553894, "loss_scale": 4.0, "retries": 0, "seconds": 160.7, "max_memory_gb": 17.11, "ce": 0.14421402267180383, "kl": 0.029270725477545056, "context": 0.15884937765076756}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.03251941129565239, "loss_scale": 4.0, "retries": 0, "seconds": 162.1, "max_memory_gb": 17.11, "ce": 0.0024692163060535677, "kl": 0.00030309064800349006, "context": 0.0026207616610918194}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.2572442591190338, "loss_scale": 4.0, "retries": 0, "seconds": 163.4, "max_memory_gb": 17.11, "ce": 0.014656258572358638, "kl": 0.0008604842732893303, "context": 0.01508650038158521}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 1.9756717681884766, "loss_scale": 4.0, "retries": 0, "seconds": 164.7, "max_memory_gb": 17.11, "ce": 0.05425937412655912, "kl": 0.31797112432715124, "context": 0.213244942802703}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 1.8622841835021973, "loss_scale": 4.0, "retries": 0, "seconds": 165.9, "max_memory_gb": 17.11, "ce": 0.26400347717572004, "kl": 0.36877443760749884, "context": 0.44839068356668577}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 0.2902776002883911, "loss_scale": 4.0, "retries": 0, "seconds": 167.1, "max_memory_gb": 17.11, "ce": 0.020944612348102964, "kl": 0.007817721256287768, "context": 0.024853473209077492}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 10.708407402038574, "loss_scale": 4.0, "retries": 0, "seconds": 168.4, "max_memory_gb": 17.11, "ce": 0.4640153396758251, "kl": 0.3734427720410167, "context": 0.6507367072044872}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.8365607261657715, "loss_scale": 4.0, "retries": 0, "seconds": 169.7, "max_memory_gb": 17.11, "ce": 0.02666767968912609, "kl": 0.020796841388801113, "context": 0.03706610045628622}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 6.578182220458984, "loss_scale": 4.0, "retries": 0, "seconds": 170.9, "max_memory_gb": 17.11, "ce": 0.29788592795375735, "kl": 0.1457981601170104, "context": 0.3707850015198346}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.29484811425209045, "loss_scale": 4.0, "retries": 0, "seconds": 172.2, "max_memory_gb": 17.11, "ce": 0.017313746095169336, "kl": 0.00676724568620557, "context": 0.02069736921112053}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 1.0555540323257446, "loss_scale": 4.0, "retries": 0, "seconds": 173.4, "max_memory_gb": 17.11, "ce": 0.08141068497207016, "kl": 0.12872518715448678, "context": 0.14577327854931355}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 0.40715789794921875, "loss_scale": 4.0, "retries": 0, "seconds": 174.6, "max_memory_gb": 17.11, "ce": 0.004128974920604378, "kl": 0.050828838843699486, "context": 0.029543393931817263}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 1.5651839971542358, "loss_scale": 4.0, "retries": 0, "seconds": 175.9, "max_memory_gb": 17.11, "ce": 0.1489327177987434, "kl": 0.04533477532095276, "context": 0.1716001117019914}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 1.0068778991699219, "loss_scale": 4.0, "retries": 0, "seconds": 177.1, "max_memory_gb": 17.11, "ce": 0.005640181079797912, "kl": 0.12044025977957062, "context": 0.06586030963808298}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.12106458842754364, "loss_scale": 4.0, "retries": 0, "seconds": 178.4, "max_memory_gb": 17.11, "ce": 0.009856824748567306, "kl": 0.004616702219038871, "context": 0.012165175576228648}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.05150427296757698, "loss_scale": 4.0, "retries": 0, "seconds": 179.6, "max_memory_gb": 17.11, "ce": 0.0042722029611468315, "kl": 0.0006831579230492935, "context": 0.004613781871739775}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.4136525392532349, "loss_scale": 4.0, "retries": 0, "seconds": 180.8, "max_memory_gb": 17.11, "ce": 0.06843968169414438, "kl": 0.023188311606645584, "context": 0.0800338381959591}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 2.0445737838745117, "loss_scale": 4.0, "retries": 0, "seconds": 182.1, "max_memory_gb": 17.11, "ce": 0.24366029209340923, "kl": 0.09264109598007053, "context": 0.2899808328365907}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 1.33491849899292, "loss_scale": 4.0, "retries": 0, "seconds": 183.3, "max_memory_gb": 17.11, "ce": 0.11705887726566289, "kl": 0.2625545136506844, "context": 0.2483361300546676}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.8857019543647766, "loss_scale": 4.0, "retries": 0, "seconds": 184.6, "max_memory_gb": 17.11, "ce": 0.0030282767984317616, "kl": 0.10741866919670429, "context": 0.05673761314392323}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.039334673434495926, "loss_scale": 4.0, "retries": 0, "seconds": 186.0, "max_memory_gb": 17.11, "ce": 0.0025784808967728168, "kl": 0.0008585110881540459, "context": 0.0030077364936005324}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 2.3551769256591797, "loss_scale": 4.0, "retries": 0, "seconds": 187.2, "max_memory_gb": 17.11, "ce": 0.08177823506412096, "kl": 0.002302613498613937, "context": 0.08292953827185556}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.21696485579013824, "loss_scale": 4.0, "retries": 0, "seconds": 188.4, "max_memory_gb": 17.11, "ce": 0.002598895283881575, "kl": 0.008241048475611024, "context": 0.006719419441651553}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.09837622940540314, "loss_scale": 4.0, "retries": 0, "seconds": 189.7, "max_memory_gb": 17.11, "ce": 0.007649110855709296, "kl": 0.000742038182579563, "context": 0.008020129789656494}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.45893529057502747, "loss_scale": 4.0, "retries": 0, "seconds": 190.9, "max_memory_gb": 17.11, "ce": 0.08677760612044949, "kl": 0.05827283882535994, "context": 0.11591402179328725}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 2.2442543506622314, "loss_scale": 4.0, "retries": 0, "seconds": 192.1, "max_memory_gb": 17.11, "ce": 0.16570081443933304, "kl": 0.02275837973866146, "context": 0.17708000092534348}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.05594692751765251, "loss_scale": 4.0, "retries": 0, "seconds": 193.5, "max_memory_gb": 17.11, "ce": 0.0021922765372437425, "kl": 0.001455521080060862, "context": 0.002920037164585665}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.125664472579956, "loss_scale": 4.0, "retries": 0, "seconds": 194.7, "max_memory_gb": 17.11, "ce": 0.003835829149466008, "kl": 0.23547172066173516, "context": 0.12157168862177059}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.1618078649044037, "loss_scale": 4.0, "retries": 0, "seconds": 196.1, "max_memory_gb": 17.11, "ce": 0.004658488760469481, "kl": 0.012884552823379636, "context": 0.01110076520126313}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.40039369463920593, "loss_scale": 4.0, "retries": 0, "seconds": 197.2, "max_memory_gb": 17.11, "ce": 0.028541448675241554, "kl": 0.007372660725195601, "context": 0.03222777874907479}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 1.2813361883163452, "loss_scale": 4.0, "retries": 0, "seconds": 198.4, "max_memory_gb": 17.11, "ce": 0.0634147995733656, "kl": 0.05045161012094468, "context": 0.0886406060308218}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 3.221086025238037, "loss_scale": 4.0, "retries": 0, "seconds": 199.7, "max_memory_gb": 17.11, "ce": 0.006240987026103539, "kl": 0.07537814279930899, "context": 0.043930057479883544}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 0.9262887835502625, "loss_scale": 4.0, "retries": 0, "seconds": 200.9, "max_memory_gb": 17.11, "ce": 0.08271665709617082, "kl": 0.08367625189384853, "context": 0.12455478316405788}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.9877274036407471, "loss_scale": 4.0, "retries": 0, "seconds": 202.1, "max_memory_gb": 17.11, "ce": 0.1001372259343043, "kl": 0.02465168322669342, "context": 0.11246306705288589}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.22807259857654572, "loss_scale": 4.0, "retries": 0, "seconds": 203.4, "max_memory_gb": 17.11, "ce": 0.0017987555911531672, "kl": 0.03219498194744119, "context": 0.017896246354212053}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.17552460730075836, "loss_scale": 4.0, "retries": 0, "seconds": 204.6, "max_memory_gb": 17.11, "ce": 0.007664782810024917, "kl": 0.009351313623483293, "context": 0.012340439716354012}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.11837141215801239, "loss_scale": 4.0, "retries": 0, "seconds": 206.0, "max_memory_gb": 17.11, "ce": 0.005324094603565754, "kl": 0.0036741179137607105, "context": 0.00716115356044611}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.43765994906425476, "loss_scale": 4.0, "retries": 0, "seconds": 207.2, "max_memory_gb": 17.11, "ce": 0.02227140578906983, "kl": 0.071832966976217, "context": 0.058187889284454286}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 2.510486364364624, "loss_scale": 4.0, "retries": 0, "seconds": 208.3, "max_memory_gb": 17.11, "ce": 0.006518037502246443, "kl": 0.08741273540363181, "context": 0.050224404731125105}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 1.297784686088562, "loss_scale": 4.0, "retries": 0, "seconds": 209.6, "max_memory_gb": 17.11, "ce": 0.02180806518299505, "kl": 0.2979441382904042, "context": 0.1707801379961893}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 3.3436713218688965, "loss_scale": 4.0, "retries": 0, "seconds": 210.8, "max_memory_gb": 17.11, "ce": 0.14370548833539942, "kl": 0.018315972221898846, "context": 0.1528634753485676}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 2.047760009765625, "loss_scale": 4.0, "retries": 0, "seconds": 212.0, "max_memory_gb": 17.11, "ce": 0.1458293676842004, "kl": 0.0270695132057881, "context": 0.15936412115115672}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.30881962180137634, "loss_scale": 4.0, "retries": 0, "seconds": 213.3, "max_memory_gb": 17.11, "ce": 0.006725224448018707, "kl": 0.015139577677473426, "context": 0.014295013272203505}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 1.1640654802322388, "loss_scale": 4.0, "retries": 0, "seconds": 214.5, "max_memory_gb": 17.11, "ce": 0.006404809169907821, "kl": 0.027806314428744372, "context": 0.02030796649341937}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 0.1955781877040863, "loss_scale": 4.0, "retries": 0, "seconds": 215.7, "max_memory_gb": 17.11, "ce": 0.012761637000949122, "kl": 0.0020585415447840205, "context": 0.013790908153168857}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.9777933359146118, "loss_scale": 4.0, "retries": 0, "seconds": 217.1, "max_memory_gb": 17.11, "ce": 0.020963935647159815, "kl": 0.2203555581800174, "context": 0.1311417119577527}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 3.4794375896453857, "loss_scale": 4.0, "retries": 0, "seconds": 218.3, "max_memory_gb": 17.11, "ce": 0.09862623403023463, "kl": 0.09894045628425374, "context": 0.14809647032961948}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.6695783138275146, "loss_scale": 4.0, "retries": 0, "seconds": 219.6, "max_memory_gb": 17.11, "ce": 0.0009058691757672932, "kl": 0.06081828864989802, "context": 0.031315013682615245}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.3305584192276, "loss_scale": 4.0, "retries": 0, "seconds": 220.7, "max_memory_gb": 17.11, "ce": 0.05272721964865923, "kl": 0.12731057188648265, "context": 0.11638250562828034}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.910356342792511, "loss_scale": 4.0, "retries": 0, "seconds": 221.9, "max_memory_gb": 17.11, "ce": 0.20163783714997408, "kl": 0.00920971589130204, "context": 0.2062427000128082}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 2.702190399169922, "loss_scale": 4.0, "retries": 0, "seconds": 223.3, "max_memory_gb": 17.11, "ce": 0.4556614537432324, "kl": 0.044401748455129564, "context": 0.47786232456564903}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 1.845682144165039, "loss_scale": 4.0, "retries": 0, "seconds": 224.5, "max_memory_gb": 17.11, "ce": 0.12268981111992616, "kl": 0.07103569559330936, "context": 0.15820766627439298}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.27194368839263916, "loss_scale": 4.0, "retries": 0, "seconds": 225.6, "max_memory_gb": 17.11, "ce": 0.009484209011134226, "kl": 0.0008856705680955201, "context": 0.00992704468080774}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.03525148332118988, "loss_scale": 4.0, "retries": 0, "seconds": 226.9, "max_memory_gb": 17.11, "ce": 0.001695061131613329, "kl": 0.00116261283892527, "context": 0.0022763675442547537}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.018516307696700096, "loss_scale": 4.0, "retries": 0, "seconds": 228.1, "max_memory_gb": 17.11, "ce": 0.0010030396770162042, "kl": 0.0005908918546992936, "context": 0.0012984856075490825}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 0.07983466982841492, "loss_scale": 4.0, "retries": 0, "seconds": 229.3, "max_memory_gb": 17.11, "ce": 0.0023461896926164627, "kl": 0.006040679640136659, "context": 0.005366529629100114}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 2.5747249126434326, "loss_scale": 4.0, "retries": 0, "seconds": 230.6, "max_memory_gb": 17.11, "ce": 0.2777603823633399, "kl": 0.03024331799133506, "context": 0.29288203719625017}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 2.577975273132324, "loss_scale": 4.0, "retries": 0, "seconds": 231.7, "max_memory_gb": 17.11, "ce": 0.07648986062849872, "kl": 0.12930377046359354, "context": 0.14114175006398}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 0.6844432950019836, "loss_scale": 4.0, "retries": 0, "seconds": 232.9, "max_memory_gb": 17.11, "ce": 0.07644126954983221, "kl": 0.020862113273551586, "context": 0.0868723238563689}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 1.1118390560150146, "loss_scale": 4.0, "retries": 0, "seconds": 234.2, "max_memory_gb": 17.11, "ce": 0.04603047307682573, "kl": 0.034970179066021956, "context": 0.06351556495792465}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.050997599959373474, "loss_scale": 4.0, "retries": 0, "seconds": 235.4, "max_memory_gb": 17.11, "ce": 0.002165908575989306, "kl": 0.001635059466934763, "context": 0.0029834384185960516}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 2.2979068756103516, "loss_scale": 4.0, "retries": 0, "seconds": 236.7, "max_memory_gb": 17.11, "ce": 0.31885785346094053, "kl": 0.508414608615567, "context": 0.5730651454650797}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.4988543689250946, "loss_scale": 4.0, "retries": 0, "seconds": 237.9, "max_memory_gb": 17.11, "ce": 0.024718775808651117, "kl": 0.08828379008741649, "context": 0.06886067108462157}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.7519648671150208, "loss_scale": 4.0, "retries": 0, "seconds": 239.1, "max_memory_gb": 17.11, "ce": 0.019686063016706612, "kl": 0.11933517458055576, "context": 0.07935365018784069}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.022477148100733757, "loss_scale": 4.0, "retries": 0, "seconds": 240.4, "max_memory_gb": 17.11, "ce": 0.001079283565559308, "kl": 0.00043121651833644137, "context": 0.001294891830184497}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.025600580498576164, "loss_scale": 4.0, "retries": 0, "seconds": 241.5, "max_memory_gb": 17.11, "ce": 0.0010450579211465083, "kl": 0.00047244269967450236, "context": 0.0012812792701879516}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.04477694630622864, "loss_scale": 4.0, "retries": 0, "seconds": 242.7, "max_memory_gb": 17.11, "ce": 0.00029041600737400586, "kl": 0.0025783261080505326, "context": 0.001579579101417039}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 1.4710042476654053, "loss_scale": 4.0, "retries": 0, "seconds": 244.0, "max_memory_gb": 17.11, "ce": 0.0007602913356095087, "kl": 0.3565833828720315, "context": 0.17905198774769815}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 2.5262362957000732, "loss_scale": 4.0, "retries": 0, "seconds": 245.3, "max_memory_gb": 17.11, "ce": 0.11349246455938555, "kl": 0.15129037074075313, "context": 0.18913764620083384}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.08117268234491348, "loss_scale": 4.0, "retries": 0, "seconds": 246.4, "max_memory_gb": 17.11, "ce": 0.006405215899576433, "kl": 0.0005103943331050687, "context": 0.0066604128369363025}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 5.427612781524658, "loss_scale": 4.0, "retries": 0, "seconds": 247.4, "max_memory_gb": 17.11, "ce": 0.1353945427981671, "kl": 0.14024257595883682, "context": 0.2055158243747428}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 1.6455399990081787, "loss_scale": 4.0, "retries": 0, "seconds": 248.4, "max_memory_gb": 17.11, "ce": 0.013062043886748143, "kl": 1.1286016878439113, "context": 0.5773628735914826}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.06784654408693314, "loss_scale": 4.0, "retries": 0, "seconds": 249.7, "max_memory_gb": 17.11, "ce": 0.0035890527160518104, "kl": 0.0037033896060165716, "context": 0.0054407475181506015}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.05887491628527641, "loss_scale": 4.0, "retries": 0, "seconds": 250.7, "max_memory_gb": 17.11, "ce": 0.0033596229914110154, "kl": 0.001981288154638605, "context": 0.004350267088739201}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 0.2991620898246765, "loss_scale": 4.0, "retries": 0, "seconds": 251.8, "max_memory_gb": 17.11, "ce": 0.004112537004402839, "kl": 0.011064195929520793, "context": 0.009644634719734313}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 1.1611900329589844, "loss_scale": 4.0, "retries": 0, "seconds": 253.0, "max_memory_gb": 17.11, "ce": 0.07261522265616804, "kl": 0.055714744768920355, "context": 0.10047259798739105}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.22135567665100098, "loss_scale": 4.0, "retries": 0, "seconds": 254.1, "max_memory_gb": 17.11, "ce": 0.00443191290833056, "kl": 0.022186880985827884, "context": 0.015525353672273923}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 2.0099430084228516, "loss_scale": 4.0, "retries": 0, "seconds": 255.1, "max_memory_gb": 17.11, "ce": 0.10656564851524308, "kl": 0.09327468066749134, "context": 0.15320299642917234}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.288331538438797, "loss_scale": 4.0, "retries": 0, "seconds": 256.4, "max_memory_gb": 17.11, "ce": 0.009099105081986636, "kl": 0.009690945344118518, "context": 0.013944577760412358}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 1.1195330619812012, "loss_scale": 4.0, "retries": 0, "seconds": 257.6, "max_memory_gb": 17.11, "ce": 0.055737669121299405, "kl": 0.028157740512540386, "context": 0.06981653852562886}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.8251088857650757, "loss_scale": 4.0, "retries": 0, "seconds": 258.9, "max_memory_gb": 17.11, "ce": 0.01867171679441526, "kl": 0.03747347067451301, "context": 0.037408452128147474}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 1.5118390321731567, "loss_scale": 4.0, "retries": 0, "seconds": 260.1, "max_memory_gb": 17.11, "ce": 0.128344296826981, "kl": 0.3240788478124159, "context": 0.2903837171252235}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 1.7639918327331543, "loss_scale": 4.0, "retries": 0, "seconds": 261.3, "max_memory_gb": 17.11, "ce": 0.3754337090358604, "kl": 0.6204467356255918, "context": 0.6856570783274947}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.20593097805976868, "loss_scale": 4.0, "retries": 0, "seconds": 262.6, "max_memory_gb": 17.11, "ce": 0.012546951387776062, "kl": 0.004183870843689874, "context": 0.014638887063483708}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.602803647518158, "loss_scale": 4.0, "retries": 0, "seconds": 263.8, "max_memory_gb": 17.11, "ce": 0.016195553180295974, "kl": 0.005799604910862399, "context": 0.01909535586310085}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.6356251239776611, "loss_scale": 4.0, "retries": 0, "seconds": 265.0, "max_memory_gb": 17.11, "ce": 0.0022931567873456515, "kl": 0.10419137008284451, "context": 0.05438884353497997}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 1.0281020402908325, "loss_scale": 4.0, "retries": 0, "seconds": 266.3, "max_memory_gb": 17.11, "ce": 0.0018325820419704542, "kl": 0.20956137336634129, "context": 0.10661326567060314}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 7.467258930206299, "loss_scale": 4.0, "retries": 0, "seconds": 267.5, "max_memory_gb": 17.11, "ce": 0.11502971399386297, "kl": 1.0916845894389553, "context": 0.6608719792711781}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.6378829479217529, "loss_scale": 4.0, "retries": 0, "seconds": 268.7, "max_memory_gb": 17.11, "ce": 0.01766970389871858, "kl": 0.02906583292860887, "context": 0.03220262041577371}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 2.3520073890686035, "loss_scale": 4.0, "retries": 0, "seconds": 270.0, "max_memory_gb": 17.11, "ce": 0.06648892851080745, "kl": 0.5035591569612734, "context": 0.3182685107458383}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 3.4367995262145996, "loss_scale": 4.0, "retries": 0, "seconds": 271.0, "max_memory_gb": 17.11, "ce": 0.2959978094586404, "kl": 0.03697266000745003, "context": 0.3144841328903567}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 1.7807588577270508, "loss_scale": 4.0, "retries": 0, "seconds": 272.0, "max_memory_gb": 17.11, "ce": 0.004950925533194095, "kl": 0.08952852790389443, "context": 0.049715188797563314}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 0.19315490126609802, "loss_scale": 4.0, "retries": 0, "seconds": 273.3, "max_memory_gb": 17.11, "ce": 0.0031933097634464502, "kl": 0.03196543476951774, "context": 0.01917602715548128}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.01620621792972088, "loss_scale": 4.0, "retries": 0, "seconds": 274.5, "max_memory_gb": 17.11, "ce": 0.0008515329318470322, "kl": 0.00017540142050620489, "context": 0.0009392336214659736}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 2.737210512161255, "loss_scale": 4.0, "retries": 0, "seconds": 275.7, "max_memory_gb": 17.11, "ce": 0.026656528338207863, "kl": 0.8636339645381668, "context": 0.4584735250391532}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.9155741930007935, "loss_scale": 4.0, "retries": 0, "seconds": 276.8, "max_memory_gb": 17.11, "ce": 0.039700193563476205, "kl": 0.22494683740660548, "context": 0.15217361506074667}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.18620958924293518, "loss_scale": 4.0, "retries": 0, "seconds": 278.0, "max_memory_gb": 17.11, "ce": 0.006266838347073644, "kl": 0.024373258109790186, "context": 0.018453467870131135}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.926576554775238, "loss_scale": 4.0, "retries": 0, "seconds": 279.4, "max_memory_gb": 17.11, "ce": 0.09435188790666871, "kl": 0.07225113050844811, "context": 0.13047745317453519}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 1.7618591785430908, "loss_scale": 4.0, "retries": 0, "seconds": 280.5, "max_memory_gb": 17.11, "ce": 0.44798231884487905, "kl": 0.017871381132863462, "context": 0.4569180231774226}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.5314323902130127, "loss_scale": 4.0, "retries": 0, "seconds": 281.8, "max_memory_gb": 17.11, "ce": 0.00440311927741277, "kl": 0.07615189623356855, "context": 0.04247906608361518}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 4.549161434173584, "loss_scale": 4.0, "retries": 0, "seconds": 282.9, "max_memory_gb": 17.11, "ce": 0.06664119045308325, "kl": 0.2554447120237455, "context": 0.19436354588106042}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.4397369623184204, "loss_scale": 4.0, "retries": 0, "seconds": 283.9, "max_memory_gb": 17.11, "ce": 0.00109259495002334, "kl": 0.056271603958521155, "context": 0.02922839629172813}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.36951324343681335, "loss_scale": 4.0, "retries": 0, "seconds": 285.1, "max_memory_gb": 17.11, "ce": 0.03135086172551382, "kl": 0.01035520957702829, "context": 0.03652846603654325}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 3.721991777420044, "loss_scale": 4.0, "retries": 0, "seconds": 286.2, "max_memory_gb": 17.11, "ce": 0.01849528989987448, "kl": 0.4366654522018507, "context": 0.23682801029644907}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 0.5299518704414368, "loss_scale": 4.0, "retries": 0, "seconds": 287.4, "max_memory_gb": 17.11, "ce": 0.052661304820503574, "kl": 0.07929374848208681, "context": 0.0923081754881423}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.6735723614692688, "loss_scale": 4.0, "retries": 0, "seconds": 288.7, "max_memory_gb": 17.11, "ce": 0.057943896972574294, "kl": 0.035463994136080146, "context": 0.07567589200334623}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 3.427743911743164, "loss_scale": 4.0, "retries": 0, "seconds": 289.7, "max_memory_gb": 17.11, "ce": 0.003442468492721673, "kl": 0.5228809831496619, "context": 0.2648829663230572}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 4.267912864685059, "loss_scale": 4.0, "retries": 0, "seconds": 290.7, "max_memory_gb": 17.11, "ce": 0.4267239849141333, "kl": 0.08787545234372374, "context": 0.47066171377082355}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.08184613287448883, "loss_scale": 4.0, "retries": 0, "seconds": 291.9, "max_memory_gb": 17.11, "ce": 0.005212594616750721, "kl": 0.0006499995256490365, "context": 0.0055375944066327065}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 3.587435245513916, "loss_scale": 4.0, "retries": 0, "seconds": 292.9, "max_memory_gb": 17.11, "ce": 0.37815351714380085, "kl": 0.2546013460669201, "context": 0.5054541893769056}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 0.4476242661476135, "loss_scale": 4.0, "retries": 0, "seconds": 293.9, "max_memory_gb": 17.11, "ce": 0.013759872104856186, "kl": 0.09420219853200251, "context": 0.060860970872454345}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 1.8216865062713623, "loss_scale": 4.0, "retries": 0, "seconds": 295.1, "max_memory_gb": 17.11, "ce": 0.2288346669083694, "kl": 0.1468874098645756, "context": 0.30227837277925573}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 2.8460545539855957, "loss_scale": 4.0, "retries": 0, "seconds": 296.1, "max_memory_gb": 17.11, "ce": 0.14876401715446264, "kl": 0.04654115581070073, "context": 0.17203459289157763}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 1.3764817714691162, "loss_scale": 4.0, "retries": 0, "seconds": 297.2, "max_memory_gb": 17.11, "ce": 0.03192761025275104, "kl": 0.08610539173332654, "context": 0.07498030821443535}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.9261974096298218, "loss_scale": 4.0, "retries": 0, "seconds": 298.3, "max_memory_gb": 17.11, "ce": 0.09479845536770881, "kl": 0.2390052385862873, "context": 0.21430106435946072}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 2.1626851558685303, "loss_scale": 4.0, "retries": 0, "seconds": 299.5, "max_memory_gb": 17.11, "ce": 0.03798800043296069, "kl": 0.12756988020555582, "context": 0.10177294118329883}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 0.11089593172073364, "loss_scale": 4.0, "retries": 0, "seconds": 300.8, "max_memory_gb": 17.11, "ce": 0.005923256641835906, "kl": 0.0029510822387237567, "context": 0.007398797868518159}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.0262637697160244, "loss_scale": 4.0, "retries": 0, "seconds": 302.0, "max_memory_gb": 17.11, "ce": 0.0019539628410711884, "kl": 0.0002386442245096987, "context": 0.00207328493706882}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.3614017367362976, "loss_scale": 4.0, "retries": 0, "seconds": 303.1, "max_memory_gb": 17.11, "ce": 0.022740101267118007, "kl": 0.00665626709087519, "context": 0.026068235223647207}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.036624472588300705, "loss_scale": 4.0, "retries": 0, "seconds": 304.3, "max_memory_gb": 17.11, "ce": 0.00179812811256852, "kl": 0.0012110023581044516, "context": 0.0024036292888922617}

runs/fixed_internvl/context/Middle8_s73/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Middle8_s73/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.14.mlp.down_proj": 144,
+    "model.language_model.layers.14.mlp.gate_proj": 144,
+    "model.language_model.layers.14.mlp.up_proj": 144,
+    "model.language_model.layers.14.self_attn.k_proj": 144,
+    "model.language_model.layers.14.self_attn.o_proj": 144,
+    "model.language_model.layers.14.self_attn.q_proj": 144,
+    "model.language_model.layers.14.self_attn.v_proj": 144,
+    "model.language_model.layers.15.mlp.down_proj": 144,
+    "model.language_model.layers.15.mlp.gate_proj": 144,
+    "model.language_model.layers.15.mlp.up_proj": 144,
+    "model.language_model.layers.15.self_attn.k_proj": 144,
+    "model.language_model.layers.15.self_attn.o_proj": 144,
+    "model.language_model.layers.15.self_attn.q_proj": 144,
+    "model.language_model.layers.15.self_attn.v_proj": 144,
+    "model.language_model.layers.16.mlp.down_proj": 144,
+    "model.language_model.layers.16.mlp.gate_proj": 144,
+    "model.language_model.layers.16.mlp.up_proj": 144,
+    "model.language_model.layers.16.self_attn.k_proj": 144,
+    "model.language_model.layers.16.self_attn.o_proj": 144,
+    "model.language_model.layers.16.self_attn.q_proj": 144,
+    "model.language_model.layers.16.self_attn.v_proj": 144,
+    "model.language_model.layers.17.mlp.down_proj": 144,
+    "model.language_model.layers.17.mlp.gate_proj": 144,
+    "model.language_model.layers.17.mlp.up_proj": 144,
+    "model.language_model.layers.17.self_attn.k_proj": 144,
+    "model.language_model.layers.17.self_attn.o_proj": 144,
+    "model.language_model.layers.17.self_attn.q_proj": 144,
+    "model.language_model.layers.17.self_attn.v_proj": 144,
+    "model.language_model.layers.18.mlp.down_proj": 144,
+    "model.language_model.layers.18.mlp.gate_proj": 144,
+    "model.language_model.layers.18.mlp.up_proj": 144,
+    "model.language_model.layers.18.self_attn.k_proj": 144,
+    "model.language_model.layers.18.self_attn.o_proj": 144,
+    "model.language_model.layers.18.self_attn.q_proj": 144,
+    "model.language_model.layers.18.self_attn.v_proj": 144,
+    "model.language_model.layers.19.mlp.down_proj": 144,
+    "model.language_model.layers.19.mlp.gate_proj": 144,
+    "model.language_model.layers.19.mlp.up_proj": 144,
+    "model.language_model.layers.19.self_attn.k_proj": 144,
+    "model.language_model.layers.19.self_attn.o_proj": 144,
+    "model.language_model.layers.19.self_attn.q_proj": 144,
+    "model.language_model.layers.19.self_attn.v_proj": 144,
+    "model.language_model.layers.20.mlp.down_proj": 144,
+    "model.language_model.layers.20.mlp.gate_proj": 144,
+    "model.language_model.layers.20.mlp.up_proj": 144,
+    "model.language_model.layers.20.self_attn.k_proj": 144,
+    "model.language_model.layers.20.self_attn.o_proj": 144,
+    "model.language_model.layers.20.self_attn.q_proj": 144,
+    "model.language_model.layers.20.self_attn.v_proj": 144,
+    "model.language_model.layers.21.mlp.down_proj": 144,
+    "model.language_model.layers.21.mlp.gate_proj": 144,
+    "model.language_model.layers.21.mlp.up_proj": 144,
+    "model.language_model.layers.21.self_attn.k_proj": 144,
+    "model.language_model.layers.21.self_attn.o_proj": 144,
+    "model.language_model.layers.21.self_attn.q_proj": 144,
+    "model.language_model.layers.21.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.14.mlp.down_proj": 72,
+    "model.language_model.layers.14.mlp.gate_proj": 72,
+    "model.language_model.layers.14.mlp.up_proj": 72,
+    "model.language_model.layers.14.self_attn.k_proj": 72,
+    "model.language_model.layers.14.self_attn.o_proj": 72,
+    "model.language_model.layers.14.self_attn.q_proj": 72,
+    "model.language_model.layers.14.self_attn.v_proj": 72,
+    "model.language_model.layers.15.mlp.down_proj": 72,
+    "model.language_model.layers.15.mlp.gate_proj": 72,
+    "model.language_model.layers.15.mlp.up_proj": 72,
+    "model.language_model.layers.15.self_attn.k_proj": 72,
+    "model.language_model.layers.15.self_attn.o_proj": 72,
+    "model.language_model.layers.15.self_attn.q_proj": 72,
+    "model.language_model.layers.15.self_attn.v_proj": 72,
+    "model.language_model.layers.16.mlp.down_proj": 72,
+    "model.language_model.layers.16.mlp.gate_proj": 72,
+    "model.language_model.layers.16.mlp.up_proj": 72,
+    "model.language_model.layers.16.self_attn.k_proj": 72,
+    "model.language_model.layers.16.self_attn.o_proj": 72,
+    "model.language_model.layers.16.self_attn.q_proj": 72,
+    "model.language_model.layers.16.self_attn.v_proj": 72,
+    "model.language_model.layers.17.mlp.down_proj": 72,
+    "model.language_model.layers.17.mlp.gate_proj": 72,
+    "model.language_model.layers.17.mlp.up_proj": 72,
+    "model.language_model.layers.17.self_attn.k_proj": 72,
+    "model.language_model.layers.17.self_attn.o_proj": 72,
+    "model.language_model.layers.17.self_attn.q_proj": 72,
+    "model.language_model.layers.17.self_attn.v_proj": 72,
+    "model.language_model.layers.18.mlp.down_proj": 72,
+    "model.language_model.layers.18.mlp.gate_proj": 72,
+    "model.language_model.layers.18.mlp.up_proj": 72,
+    "model.language_model.layers.18.self_attn.k_proj": 72,
+    "model.language_model.layers.18.self_attn.o_proj": 72,
+    "model.language_model.layers.18.self_attn.q_proj": 72,
+    "model.language_model.layers.18.self_attn.v_proj": 72,
+    "model.language_model.layers.19.mlp.down_proj": 72,
+    "model.language_model.layers.19.mlp.gate_proj": 72,
+    "model.language_model.layers.19.mlp.up_proj": 72,
+    "model.language_model.layers.19.self_attn.k_proj": 72,
+    "model.language_model.layers.19.self_attn.o_proj": 72,
+    "model.language_model.layers.19.self_attn.q_proj": 72,
+    "model.language_model.layers.19.self_attn.v_proj": 72,
+    "model.language_model.layers.20.mlp.down_proj": 72,
+    "model.language_model.layers.20.mlp.gate_proj": 72,
+    "model.language_model.layers.20.mlp.up_proj": 72,
+    "model.language_model.layers.20.self_attn.k_proj": 72,
+    "model.language_model.layers.20.self_attn.o_proj": 72,
+    "model.language_model.layers.20.self_attn.q_proj": 72,
+    "model.language_model.layers.20.self_attn.v_proj": 72,
+    "model.language_model.layers.21.mlp.down_proj": 72,
+    "model.language_model.layers.21.mlp.gate_proj": 72,
+    "model.language_model.layers.21.mlp.up_proj": 72,
+    "model.language_model.layers.21.self_attn.k_proj": 72,
+    "model.language_model.layers.21.self_attn.o_proj": 72,
+    "model.language_model.layers.21.self_attn.q_proj": 72,
+    "model.language_model.layers.21.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.17.self_attn.o_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.14.mlp.gate_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.17.self_attn.v_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.20.self_attn.v_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.14.mlp.up_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.q_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.14.mlp.down_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.16.self_attn.k_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.14.self_attn.v_proj",
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.14.self_attn.k_proj",
+    "model.language_model.layers.14.self_attn.o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Middle8_s73/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 73,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Middle8",
+    "layers": [
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21
+    ],
+    "ranks": {
+      "14": 72,
+      "15": 72,
+      "16": 72,
+      "17": 72,
+      "18": 72,
+      "19": 72,
+      "20": 72,
+      "21": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Middle8_s73/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 16.388710021972656, "loss_scale": 4.0, "retries": 0, "seconds": 2.2, "max_memory_gb": 16.79, "ce": 0.6797246597125195, "kl": 0.0438283761832281, "context": 0.7016388622396335}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 29.736902236938477, "loss_scale": 4.0, "retries": 0, "seconds": 3.5, "max_memory_gb": 17.1, "ce": 1.678076982498169, "kl": 0.697758472058922, "context": 2.026956260204315}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 11.799981117248535, "loss_scale": 4.0, "retries": 0, "seconds": 4.7, "max_memory_gb": 17.1, "ce": 0.4536920292885043, "kl": 0.11318458517780527, "context": 0.510284312418662}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 8.105321884155273, "loss_scale": 4.0, "retries": 0, "seconds": 5.8, "max_memory_gb": 17.1, "ce": 0.3281868137419224, "kl": 0.021149163691006834, "context": 0.3387613917584531}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 5.230127334594727, "loss_scale": 4.0, "retries": 0, "seconds": 7.0, "max_memory_gb": 17.11, "ce": 0.10543392399631557, "kl": 0.1283172114053741, "context": 0.16959253046661615}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 2.935256004333496, "loss_scale": 4.0, "retries": 0, "seconds": 8.0, "max_memory_gb": 17.11, "ce": 0.18326205760240555, "kl": 0.05080372653901577, "context": 0.20866392040625215}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 0.23315435647964478, "loss_scale": 4.0, "retries": 0, "seconds": 9.1, "max_memory_gb": 17.11, "ce": 0.016528971686057048, "kl": 0.009539819874589739, "context": 0.021298882686096476}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 2.087131977081299, "loss_scale": 4.0, "retries": 0, "seconds": 10.3, "max_memory_gb": 17.11, "ce": 0.3416635378089268, "kl": 0.4796222451259382, "context": 0.581474659673404}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 3.3483829498291016, "loss_scale": 4.0, "retries": 0, "seconds": 11.4, "max_memory_gb": 17.11, "ce": 0.04310709494166076, "kl": 0.5093184667930473, "context": 0.2977663262281567}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 2.114439010620117, "loss_scale": 4.0, "retries": 0, "seconds": 12.7, "max_memory_gb": 17.11, "ce": 0.03540570873883553, "kl": 0.3676521247095934, "context": 0.21923176632117247}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.38529205322265625, "loss_scale": 4.0, "retries": 0, "seconds": 13.9, "max_memory_gb": 17.11, "ce": 0.011438455083407462, "kl": 0.09196965387673117, "context": 0.05742328194901347}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 3.454503059387207, "loss_scale": 4.0, "retries": 0, "seconds": 15.1, "max_memory_gb": 17.11, "ce": 0.03603966300670436, "kl": 0.03120008736368618, "context": 0.051639706656715134}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 2.187765121459961, "loss_scale": 4.0, "retries": 0, "seconds": 16.3, "max_memory_gb": 17.11, "ce": 0.1514721477724379, "kl": 0.08245024671487045, "context": 0.19269727167556994}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 1.0278252363204956, "loss_scale": 4.0, "retries": 0, "seconds": 17.5, "max_memory_gb": 17.11, "ce": 0.04994309414996678, "kl": 0.07921072526369244, "context": 0.08954845303378534}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 8.185558319091797, "loss_scale": 4.0, "retries": 0, "seconds": 18.7, "max_memory_gb": 17.11, "ce": 0.13290882261935622, "kl": 0.1524584830040112, "context": 0.20913806330645457}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 7.114831447601318, "loss_scale": 4.0, "retries": 0, "seconds": 19.9, "max_memory_gb": 17.11, "ce": 0.7706449560355395, "kl": 0.05069277044003684, "context": 0.7959913610102376}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 3.9933416843414307, "loss_scale": 4.0, "retries": 0, "seconds": 21.1, "max_memory_gb": 17.11, "ce": 0.08066581329330802, "kl": 0.4104422299860744, "context": 0.2858869371993933}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 2.3597869873046875, "loss_scale": 4.0, "retries": 0, "seconds": 22.3, "max_memory_gb": 17.11, "ce": 0.002166071644751355, "kl": 0.27427605412958656, "context": 0.13930410152534023}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 3.2376770973205566, "loss_scale": 4.0, "retries": 0, "seconds": 23.5, "max_memory_gb": 17.11, "ce": 0.24431840109900804, "kl": 0.23856226657017032, "context": 0.36359952507336857}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 1.0177828073501587, "loss_scale": 4.0, "retries": 0, "seconds": 24.7, "max_memory_gb": 17.11, "ce": 0.07863808883848833, "kl": 0.31777287953332234, "context": 0.23752452767075738}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.10525301098823547, "loss_scale": 4.0, "retries": 0, "seconds": 25.8, "max_memory_gb": 17.11, "ce": 0.008009995304746553, "kl": 0.0012650043063331395, "context": 0.00864249731239397}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 0.8302364349365234, "loss_scale": 4.0, "retries": 0, "seconds": 27.1, "max_memory_gb": 17.11, "ce": 0.16901481663808227, "kl": 0.2877459402079694, "context": 0.312887791544199}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 3.0152132511138916, "loss_scale": 4.0, "retries": 0, "seconds": 28.1, "max_memory_gb": 17.11, "ce": 0.2200648643920431, "kl": 0.1043061744858278, "context": 0.2722179499978665}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 2.317506790161133, "loss_scale": 4.0, "retries": 0, "seconds": 29.1, "max_memory_gb": 17.11, "ce": 0.14220194332301617, "kl": 0.22395865322323516, "context": 0.25418127176817507}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 1.8465347290039062, "loss_scale": 4.0, "retries": 0, "seconds": 30.2, "max_memory_gb": 17.11, "ce": 0.09620132140116766, "kl": 0.0609467207104899, "context": 0.1266746788751334}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 6.372828006744385, "loss_scale": 4.0, "retries": 0, "seconds": 31.2, "max_memory_gb": 17.11, "ce": 0.2621262550528627, "kl": 0.4563656758930392, "context": 0.4903090929874452}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 9.116400718688965, "loss_scale": 4.0, "retries": 0, "seconds": 32.3, "max_memory_gb": 17.11, "ce": 0.20349478903517593, "kl": 0.24543434183760837, "context": 0.32621196741820313}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 0.07593002915382385, "loss_scale": 4.0, "retries": 0, "seconds": 33.3, "max_memory_gb": 17.11, "ce": 0.004226837365422398, "kl": 0.0009534243454254465, "context": 0.004703549493569881}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 0.28701165318489075, "loss_scale": 4.0, "retries": 0, "seconds": 34.2, "max_memory_gb": 17.11, "ce": 0.01163880864623934, "kl": 0.007136598927900195, "context": 0.015207108343020082}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 2.4493343830108643, "loss_scale": 4.0, "retries": 0, "seconds": 35.3, "max_memory_gb": 17.11, "ce": 0.04179166560061276, "kl": 0.047989044636779, "context": 0.0657861897489056}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 1.1590114831924438, "loss_scale": 4.0, "retries": 0, "seconds": 36.3, "max_memory_gb": 17.11, "ce": 0.01152442637248896, "kl": 0.5581300859630574, "context": 0.2905894760042429}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 5.709082126617432, "loss_scale": 4.0, "retries": 0, "seconds": 37.3, "max_memory_gb": 17.11, "ce": 0.17309104454761837, "kl": 0.4778371529537253, "context": 0.41200962103903294}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 0.24931907653808594, "loss_scale": 4.0, "retries": 0, "seconds": 38.4, "max_memory_gb": 17.11, "ce": 0.017500904388725758, "kl": 0.031768264292622916, "context": 0.03338503697887063}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 0.6452651619911194, "loss_scale": 4.0, "retries": 0, "seconds": 39.4, "max_memory_gb": 17.11, "ce": 0.03773903951514512, "kl": 0.2187655947636813, "context": 0.14712184248492122}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 2.4293577671051025, "loss_scale": 4.0, "retries": 0, "seconds": 40.5, "max_memory_gb": 17.11, "ce": 0.04382029105909169, "kl": 0.22750340198854246, "context": 0.15757198841311038}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 0.4901614785194397, "loss_scale": 4.0, "retries": 0, "seconds": 41.4, "max_memory_gb": 17.11, "ce": 0.061516904504969716, "kl": 0.019571429875213653, "context": 0.0713026209268719}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 0.27837809920310974, "loss_scale": 4.0, "retries": 0, "seconds": 42.4, "max_memory_gb": 17.11, "ce": 0.0199778713285923, "kl": 0.011490491451695561, "context": 0.025723117403686047}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 0.6805503368377686, "loss_scale": 4.0, "retries": 0, "seconds": 43.5, "max_memory_gb": 17.11, "ce": 0.054543907346669585, "kl": 0.00830342500921688, "context": 0.058695618412457407}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 2.2483739852905273, "loss_scale": 4.0, "retries": 0, "seconds": 44.5, "max_memory_gb": 17.11, "ce": 0.3350136557710357, "kl": 0.02409297941994737, "context": 0.3470601587323472}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.3566892147064209, "loss_scale": 4.0, "retries": 0, "seconds": 45.5, "max_memory_gb": 17.11, "ce": 0.020120870729442686, "kl": 0.06734268052969128, "context": 0.05379221122711897}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 0.47807028889656067, "loss_scale": 4.0, "retries": 0, "seconds": 46.6, "max_memory_gb": 17.11, "ce": 0.057042850530706346, "kl": 0.0027067012051702477, "context": 0.05839620006736368}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 1.2788364887237549, "loss_scale": 4.0, "retries": 0, "seconds": 47.6, "max_memory_gb": 17.11, "ce": 0.053659280558349565, "kl": 0.045357631857768865, "context": 0.07633809893741272}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 1.5907044410705566, "loss_scale": 4.0, "retries": 0, "seconds": 48.6, "max_memory_gb": 17.11, "ce": 0.2929852055385709, "kl": 0.13699238654226065, "context": 0.3614814067259431}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 0.17363597452640533, "loss_scale": 4.0, "retries": 0, "seconds": 49.7, "max_memory_gb": 17.11, "ce": 0.009903272788506001, "kl": 0.003125437557173427, "context": 0.011465991789009422}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 1.379949688911438, "loss_scale": 4.0, "retries": 0, "seconds": 50.6, "max_memory_gb": 17.11, "ce": 0.029218317824415863, "kl": 0.5973503682762384, "context": 0.32789349369704723}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 2.320235013961792, "loss_scale": 4.0, "retries": 0, "seconds": 51.7, "max_memory_gb": 17.11, "ce": 0.013696565307327546, "kl": 0.7345590954946601, "context": 0.3809761246375274}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.15512336790561676, "loss_scale": 4.0, "retries": 0, "seconds": 52.7, "max_memory_gb": 17.11, "ce": 0.01188661073683761, "kl": 0.004287193340132944, "context": 0.014030207530595362}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.16475726664066315, "loss_scale": 4.0, "retries": 0, "seconds": 53.7, "max_memory_gb": 17.11, "ce": 0.012551628300570883, "kl": 0.006487559174274793, "context": 0.01579540834063664}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 1.4569060802459717, "loss_scale": 4.0, "retries": 0, "seconds": 54.8, "max_memory_gb": 17.11, "ce": 0.04599633230827749, "kl": 0.03514148770773318, "context": 0.0635670778574422}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.10826465487480164, "loss_scale": 4.0, "retries": 0, "seconds": 55.8, "max_memory_gb": 17.11, "ce": 0.0038410754059441388, "kl": 0.0053688960542785935, "context": 0.006525523436721414}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 3.0806102752685547, "loss_scale": 4.0, "retries": 0, "seconds": 56.8, "max_memory_gb": 17.11, "ce": 0.09654662772663869, "kl": 0.11641570836945903, "context": 0.15475448197685182}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 4.502801895141602, "loss_scale": 4.0, "retries": 0, "seconds": 57.8, "max_memory_gb": 17.11, "ce": 0.2700261506251991, "kl": 0.2885586123011308, "context": 0.41430545691400766}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 2.971177577972412, "loss_scale": 4.0, "retries": 0, "seconds": 58.8, "max_memory_gb": 17.11, "ce": 0.450048932703794, "kl": 0.37825113313738257, "context": 0.6391745081637055}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 2.3325488567352295, "loss_scale": 4.0, "retries": 0, "seconds": 59.8, "max_memory_gb": 17.11, "ce": 0.28117986395955086, "kl": 0.12493042652204167, "context": 0.34364507417194545}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 11.447566986083984, "loss_scale": 4.0, "retries": 0, "seconds": 60.9, "max_memory_gb": 17.11, "ce": 0.1755229877308011, "kl": 0.10469274912611581, "context": 0.2278693625703454}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.4887092709541321, "loss_scale": 4.0, "retries": 0, "seconds": 61.9, "max_memory_gb": 17.11, "ce": 0.010786993778310716, "kl": 0.10216637189296307, "context": 0.061870179837569594}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 1.2662327289581299, "loss_scale": 4.0, "retries": 0, "seconds": 62.9, "max_memory_gb": 17.11, "ce": 0.0392429953062674, "kl": 0.617174528677424, "context": 0.3478302643634379}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.4377374053001404, "loss_scale": 4.0, "retries": 0, "seconds": 64.0, "max_memory_gb": 17.11, "ce": 0.005215011231484823, "kl": 0.14761400316638174, "context": 0.07902201369870454}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 0.25751134753227234, "loss_scale": 4.0, "retries": 0, "seconds": 64.9, "max_memory_gb": 17.11, "ce": 0.01572899060556665, "kl": 0.007283136830665171, "context": 0.019370558788068593}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 1.2161519527435303, "loss_scale": 4.0, "retries": 0, "seconds": 66.1, "max_memory_gb": 17.11, "ce": 0.05305232771206647, "kl": 0.03702066600089893, "context": 0.07156266155652702}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 4.800438404083252, "loss_scale": 4.0, "retries": 0, "seconds": 67.0, "max_memory_gb": 17.11, "ce": 0.27754649287089705, "kl": 0.24836887666606344, "context": 0.40173094044439495}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 2.916520118713379, "loss_scale": 4.0, "retries": 0, "seconds": 68.0, "max_memory_gb": 17.11, "ce": 0.24727981485193595, "kl": 0.06572797380795237, "context": 0.2801437965827063}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 0.4836809039115906, "loss_scale": 4.0, "retries": 0, "seconds": 69.1, "max_memory_gb": 17.11, "ce": 0.05064494837279199, "kl": 0.014826972601440502, "context": 0.05805843372945674}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.8109373450279236, "loss_scale": 4.0, "retries": 0, "seconds": 70.1, "max_memory_gb": 17.11, "ce": 0.12285064609022811, "kl": 0.11288906913250685, "context": 0.17929518036544323}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 2.000115394592285, "loss_scale": 4.0, "retries": 0, "seconds": 71.1, "max_memory_gb": 17.11, "ce": 0.07714580371975899, "kl": 0.009185164992231876, "context": 0.08173838793300092}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 0.20087464153766632, "loss_scale": 4.0, "retries": 0, "seconds": 72.2, "max_memory_gb": 17.11, "ce": 0.008711376111023128, "kl": 0.003262977916165255, "context": 0.010342865047277883}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 2.1949892044067383, "loss_scale": 4.0, "retries": 0, "seconds": 73.2, "max_memory_gb": 17.11, "ce": 0.2717333072796464, "kl": 0.0228862541844137, "context": 0.2831764293368906}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.6253729462623596, "loss_scale": 4.0, "retries": 0, "seconds": 74.3, "max_memory_gb": 17.11, "ce": 0.03989228792488575, "kl": 0.02206368392216973, "context": 0.050924129551276565}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 0.9882404804229736, "loss_scale": 4.0, "retries": 0, "seconds": 75.2, "max_memory_gb": 17.11, "ce": 0.06235567547264509, "kl": 0.12082984482185566, "context": 0.12277059478219599}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.29462701082229614, "loss_scale": 4.0, "retries": 0, "seconds": 76.2, "max_memory_gb": 17.11, "ce": 0.01580407231085701, "kl": 0.029347959136885038, "context": 0.030478051005047746}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.19353559613227844, "loss_scale": 4.0, "retries": 0, "seconds": 77.4, "max_memory_gb": 17.11, "ce": 0.0019981368095614016, "kl": 0.01615505495283287, "context": 0.01007566397311166}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.23732805252075195, "loss_scale": 4.0, "retries": 0, "seconds": 78.3, "max_memory_gb": 17.11, "ce": 0.01000569031748455, "kl": 0.008203232871892396, "context": 0.014107307215454057}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 2.1165473461151123, "loss_scale": 4.0, "retries": 0, "seconds": 79.3, "max_memory_gb": 17.11, "ce": 0.02226620366127463, "kl": 0.17732604492971404, "context": 0.11092922612442635}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.8179605603218079, "loss_scale": 4.0, "retries": 0, "seconds": 80.4, "max_memory_gb": 17.11, "ce": 0.07440181247147848, "kl": 0.047935270940797636, "context": 0.09836944794369629}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.2640820741653442, "loss_scale": 4.0, "retries": 0, "seconds": 81.4, "max_memory_gb": 17.11, "ce": 0.0064143639710891875, "kl": 0.09364849047233292, "context": 0.05323860910721123}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 2.0962228775024414, "loss_scale": 4.0, "retries": 0, "seconds": 82.4, "max_memory_gb": 17.11, "ce": 0.4688120377031737, "kl": 0.04074832629839875, "context": 0.4891862064432644}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.4410758912563324, "loss_scale": 4.0, "retries": 0, "seconds": 83.5, "max_memory_gb": 17.11, "ce": 0.0014502264020848088, "kl": 0.05154130927257938, "context": 0.027220881413086317}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 1.3806967735290527, "loss_scale": 4.0, "retries": 0, "seconds": 84.5, "max_memory_gb": 17.11, "ce": 0.0010993446085194591, "kl": 0.8235735948262572, "context": 0.4128861526878609}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.5669435262680054, "loss_scale": 4.0, "retries": 0, "seconds": 85.6, "max_memory_gb": 17.11, "ce": 0.015755253960378468, "kl": 0.009630688175093383, "context": 0.02057059807702899}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 2.789320468902588, "loss_scale": 4.0, "retries": 0, "seconds": 86.6, "max_memory_gb": 17.11, "ce": 0.20600070764976408, "kl": 0.057817715560531724, "context": 0.2349095672925614}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 2.241476535797119, "loss_scale": 4.0, "retries": 0, "seconds": 87.6, "max_memory_gb": 17.11, "ce": 0.14233355971009587, "kl": 0.06500303791938222, "context": 0.17483508161330974}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.3598007559776306, "loss_scale": 4.0, "retries": 0, "seconds": 88.7, "max_memory_gb": 17.11, "ce": 0.01360347360605374, "kl": 0.009404023135175521, "context": 0.018305485136806965}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 0.19331856071949005, "loss_scale": 4.0, "retries": 0, "seconds": 89.7, "max_memory_gb": 17.11, "ce": 0.003390253748420946, "kl": 0.013211658153124972, "context": 0.009996082814268448}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 1.7551424503326416, "loss_scale": 4.0, "retries": 0, "seconds": 90.7, "max_memory_gb": 17.11, "ce": 0.000530365753093065, "kl": 0.2583249691931542, "context": 0.12969285275721631}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.17389002442359924, "loss_scale": 4.0, "retries": 0, "seconds": 91.8, "max_memory_gb": 17.11, "ce": 0.005312299213755978, "kl": 0.005179559674303391, "context": 0.007902079052655608}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 1.0112297534942627, "loss_scale": 4.0, "retries": 0, "seconds": 92.8, "max_memory_gb": 17.11, "ce": 0.0017156992500986235, "kl": 0.5557204688816455, "context": 0.2795759329928842}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 0.09932710975408554, "loss_scale": 4.0, "retries": 0, "seconds": 93.8, "max_memory_gb": 17.11, "ce": 0.004326405183746829, "kl": 0.007721594610757165, "context": 0.008187202511180658}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.9940993785858154, "loss_scale": 4.0, "retries": 0, "seconds": 94.9, "max_memory_gb": 17.11, "ce": 0.0251889146384201, "kl": 0.04592199573698963, "context": 0.04814991441890015}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 2.4417219161987305, "loss_scale": 4.0, "retries": 0, "seconds": 95.9, "max_memory_gb": 17.11, "ce": 0.08893569807696622, "kl": 0.14543655831539581, "context": 0.1616539772367105}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.6952576637268066, "loss_scale": 4.0, "retries": 0, "seconds": 96.9, "max_memory_gb": 17.11, "ce": 0.04595611208060291, "kl": 0.05847249971338897, "context": 0.07519235822110204}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 3.1749536991119385, "loss_scale": 4.0, "retries": 0, "seconds": 97.9, "max_memory_gb": 17.11, "ce": 0.006595998536795378, "kl": 0.2579554311669199, "context": 0.13557370868511498}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 0.9721628427505493, "loss_scale": 4.0, "retries": 0, "seconds": 98.9, "max_memory_gb": 17.11, "ce": 0.043080119212390855, "kl": 0.12432249621997471, "context": 0.1052413649886148}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 2.1703908443450928, "loss_scale": 4.0, "retries": 0, "seconds": 100.0, "max_memory_gb": 17.11, "ce": 0.20853915213956498, "kl": 0.3491960554383695, "context": 0.3831371832638979}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 0.06697626411914825, "loss_scale": 4.0, "retries": 0, "seconds": 101.0, "max_memory_gb": 17.11, "ce": 0.00374541890778346, "kl": 0.0016010601329981, "context": 0.004545948977465741}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 3.2132325172424316, "loss_scale": 4.0, "retries": 0, "seconds": 102.0, "max_memory_gb": 17.11, "ce": 0.21780223144742195, "kl": 0.6019340563725564, "context": 0.5187692414037883}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 0.5206831693649292, "loss_scale": 4.0, "retries": 0, "seconds": 103.1, "max_memory_gb": 17.11, "ce": 0.023528513451310573, "kl": 0.062174161605071276, "context": 0.054615594912320375}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.8977774977684021, "loss_scale": 4.0, "retries": 0, "seconds": 104.0, "max_memory_gb": 17.11, "ce": 0.017401611112290993, "kl": 0.028634267335291952, "context": 0.031718743848614395}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.5014281868934631, "loss_scale": 4.0, "retries": 0, "seconds": 105.0, "max_memory_gb": 17.11, "ce": 0.02680706512182951, "kl": 0.020919981972838286, "context": 0.03726705559529364}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 1.62566077709198, "loss_scale": 4.0, "retries": 0, "seconds": 106.1, "max_memory_gb": 17.11, "ce": 0.08804539806442335, "kl": 0.0709752947313973, "context": 0.12353304168209434}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.9967105388641357, "loss_scale": 4.0, "retries": 0, "seconds": 107.1, "max_memory_gb": 17.11, "ce": 0.05730841326294467, "kl": 0.07157309600734152, "context": 0.09309496311470866}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 2.496903419494629, "loss_scale": 4.0, "retries": 0, "seconds": 108.2, "max_memory_gb": 17.11, "ce": 0.24618733575334772, "kl": 0.030825096655462403, "context": 0.2615998875699006}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 2.753343343734741, "loss_scale": 4.0, "retries": 0, "seconds": 109.2, "max_memory_gb": 17.11, "ce": 0.1718621529871598, "kl": 0.03474488378742535, "context": 0.1892345915257465}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.08106186240911484, "loss_scale": 4.0, "retries": 0, "seconds": 110.2, "max_memory_gb": 17.11, "ce": 0.004527656841673888, "kl": 0.0034912712762888987, "context": 0.006273292179685086}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 1.0819649696350098, "loss_scale": 4.0, "retries": 0, "seconds": 111.3, "max_memory_gb": 17.11, "ce": 0.1432647114561405, "kl": 0.012043504568282515, "context": 0.14928645850159228}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 2.510498523712158, "loss_scale": 4.0, "retries": 0, "seconds": 112.3, "max_memory_gb": 17.11, "ce": 0.17714639706537127, "kl": 0.11234393110498786, "context": 0.23331836075522006}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.767024040222168, "loss_scale": 4.0, "retries": 0, "seconds": 113.2, "max_memory_gb": 17.11, "ce": 0.013571598334237933, "kl": 0.12446070104488172, "context": 0.07580194575712085}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.10563190281391144, "loss_scale": 4.0, "retries": 0, "seconds": 114.4, "max_memory_gb": 17.11, "ce": 0.0016948412085184827, "kl": 0.008751491472594353, "context": 0.0060705869109369814}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 0.19934611022472382, "loss_scale": 4.0, "retries": 0, "seconds": 115.3, "max_memory_gb": 17.11, "ce": 0.014123933855444193, "kl": 0.005922582800849341, "context": 0.017085225321352482}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 0.11821366101503372, "loss_scale": 4.0, "retries": 0, "seconds": 116.3, "max_memory_gb": 17.11, "ce": 0.003171029209624976, "kl": 0.013050293200649321, "context": 0.009696175926364958}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.2130143940448761, "loss_scale": 4.0, "retries": 0, "seconds": 117.4, "max_memory_gb": 17.11, "ce": 0.005741128115914762, "kl": 0.0023520261675002985, "context": 0.006917141028679907}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 1.7132281064987183, "loss_scale": 4.0, "retries": 0, "seconds": 118.4, "max_memory_gb": 17.11, "ce": 0.07771702122408897, "kl": 0.0013903363396821078, "context": 0.07841219077818096}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.4611670672893524, "loss_scale": 4.0, "retries": 0, "seconds": 119.5, "max_memory_gb": 17.11, "ce": 0.03641211855574511, "kl": 0.0347271286555042, "context": 0.053775684733409435}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 3.419224262237549, "loss_scale": 4.0, "retries": 0, "seconds": 120.5, "max_memory_gb": 17.11, "ce": 0.6469701845344389, "kl": 0.061993453457034775, "context": 0.6779668936505914}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 1.4129990339279175, "loss_scale": 4.0, "retries": 0, "seconds": 121.5, "max_memory_gb": 17.11, "ce": 0.15102994430344552, "kl": 0.10692097194259986, "context": 0.20449043216649443}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 0.38416266441345215, "loss_scale": 4.0, "retries": 0, "seconds": 122.7, "max_memory_gb": 17.11, "ce": 0.023911054682685062, "kl": 0.009448645106630238, "context": 0.028635376365855336}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 4.244604587554932, "loss_scale": 4.0, "retries": 0, "seconds": 123.7, "max_memory_gb": 17.11, "ce": 0.41095255565596744, "kl": 0.014316949440399185, "context": 0.4181110409554094}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 1.4932472705841064, "loss_scale": 4.0, "retries": 0, "seconds": 124.8, "max_memory_gb": 17.11, "ce": 0.011577456563827582, "kl": 0.02024305472150445, "context": 0.021698984623071738}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.00172758102417, "loss_scale": 4.0, "retries": 0, "seconds": 125.9, "max_memory_gb": 17.11, "ce": 0.029695469420403242, "kl": 0.02057315269485116, "context": 0.039982046000659466}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.09550952166318893, "loss_scale": 4.0, "retries": 0, "seconds": 126.9, "max_memory_gb": 17.11, "ce": 0.005219395461608656, "kl": 0.0003676122528304404, "context": 0.0054032016269047745}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 2.1426031589508057, "loss_scale": 4.0, "retries": 0, "seconds": 127.9, "max_memory_gb": 17.11, "ce": 0.13915803446434438, "kl": 0.17970782937482, "context": 0.2290119482204318}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 2.7187302112579346, "loss_scale": 4.0, "retries": 0, "seconds": 129.0, "max_memory_gb": 17.11, "ce": 0.009828556969296187, "kl": 0.6783388916810509, "context": 0.34899800701532513}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.6071298122406006, "loss_scale": 4.0, "retries": 0, "seconds": 130.0, "max_memory_gb": 17.11, "ce": 0.04309904928231845, "kl": 0.004407175997585, "context": 0.04530263561173342}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 6.046116352081299, "loss_scale": 4.0, "retries": 0, "seconds": 131.0, "max_memory_gb": 17.11, "ce": 0.33798588991339784, "kl": 0.06469255380216055, "context": 0.3703321677458007}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 0.09836559742689133, "loss_scale": 4.0, "retries": 0, "seconds": 132.1, "max_memory_gb": 17.11, "ce": 0.00084365886868909, "kl": 0.015557029550109291, "context": 0.008622173656476662}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 3.943025827407837, "loss_scale": 4.0, "retries": 0, "seconds": 133.1, "max_memory_gb": 17.11, "ce": 0.19227072736248374, "kl": 1.306954264640808, "context": 0.8457478508353233}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 0.9511474370956421, "loss_scale": 4.0, "retries": 0, "seconds": 134.2, "max_memory_gb": 17.11, "ce": 0.025832059473032132, "kl": 0.03806166731737903, "context": 0.04486289336637128}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.02964058332145214, "loss_scale": 4.0, "retries": 0, "seconds": 135.2, "max_memory_gb": 17.11, "ce": 0.0018884360324591398, "kl": 0.0014753439108972088, "context": 0.002626108020194806}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 0.19016848504543304, "loss_scale": 4.0, "retries": 0, "seconds": 136.2, "max_memory_gb": 17.11, "ce": 0.006902823632117361, "kl": 0.03650914004538208, "context": 0.025157393189147115}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 5.605736255645752, "loss_scale": 4.0, "retries": 0, "seconds": 137.3, "max_memory_gb": 17.11, "ce": 0.10180742534430465, "kl": 0.06611247366527095, "context": 0.13486365984135773}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 3.3917737007141113, "loss_scale": 4.0, "retries": 0, "seconds": 138.3, "max_memory_gb": 17.11, "ce": 0.09772127121686935, "kl": 0.13501700104097836, "context": 0.16522977175191045}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 2.9585037231445312, "loss_scale": 4.0, "retries": 0, "seconds": 139.2, "max_memory_gb": 17.11, "ce": 0.10063231734966394, "kl": 0.2583120206104468, "context": 0.22978832762601087}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 0.4548414647579193, "loss_scale": 4.0, "retries": 0, "seconds": 140.3, "max_memory_gb": 17.11, "ce": 0.021394508657976985, "kl": 0.015046442960738204, "context": 0.02891772915609181}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 0.45712271332740784, "loss_scale": 4.0, "retries": 0, "seconds": 141.3, "max_memory_gb": 17.11, "ce": 0.0469851262823795, "kl": 0.072375174900003, "context": 0.08317271374107804}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.14400556683540344, "loss_scale": 4.0, "retries": 0, "seconds": 142.4, "max_memory_gb": 17.11, "ce": 0.004020673659397289, "kl": 0.0018113034702764708, "context": 0.004926325433189049}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 6.055003643035889, "loss_scale": 4.0, "retries": 0, "seconds": 143.4, "max_memory_gb": 17.11, "ce": 0.45632991462480277, "kl": 0.0007990239482751349, "context": 0.4567294163425686}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 8.454487800598145, "loss_scale": 4.0, "retries": 0, "seconds": 144.4, "max_memory_gb": 17.11, "ce": 0.8216664307110477, "kl": 0.3934934533026535, "context": 1.018413164827507}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 7.335193157196045, "loss_scale": 4.0, "retries": 0, "seconds": 145.5, "max_memory_gb": 17.11, "ce": 0.3040545973344706, "kl": 0.007975090444233501, "context": 0.30804214806994423}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 0.299898624420166, "loss_scale": 4.0, "retries": 0, "seconds": 146.5, "max_memory_gb": 17.11, "ce": 0.011041638033930212, "kl": 0.00815175928073586, "context": 0.015117518094484694}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 0.06919125467538834, "loss_scale": 4.0, "retries": 0, "seconds": 147.5, "max_memory_gb": 17.11, "ce": 0.0036223860370228067, "kl": 0.008017613188712858, "context": 0.0076311929151415825}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.4954937994480133, "loss_scale": 4.0, "retries": 0, "seconds": 148.6, "max_memory_gb": 17.11, "ce": 0.051746508688665926, "kl": 0.00336074586630275, "context": 0.05342688114615157}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.08416060358285904, "loss_scale": 4.0, "retries": 0, "seconds": 149.6, "max_memory_gb": 17.11, "ce": 0.003975406820245553, "kl": 0.00731662448743009, "context": 0.0076337190039339475}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 1.1164613962173462, "loss_scale": 4.0, "retries": 0, "seconds": 150.5, "max_memory_gb": 17.11, "ce": 0.15748833172256127, "kl": 0.06015970205771737, "context": 0.18756818101974204}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 2.9136030673980713, "loss_scale": 4.0, "retries": 0, "seconds": 151.6, "max_memory_gb": 17.11, "ce": 0.11387897207168862, "kl": 0.51096680747969, "context": 0.3693623614963144}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.13884206116199493, "loss_scale": 4.0, "retries": 0, "seconds": 152.6, "max_memory_gb": 17.11, "ce": 0.007608698433614336, "kl": 0.01673027186143372, "context": 0.0159738345828373}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 1.2884442806243896, "loss_scale": 4.0, "retries": 0, "seconds": 153.7, "max_memory_gb": 17.11, "ce": 0.12316139609902166, "kl": 0.21373166001285426, "context": 0.23002722638193518}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 2.8596582412719727, "loss_scale": 4.0, "retries": 0, "seconds": 154.7, "max_memory_gb": 17.11, "ce": 0.2708317408687435, "kl": 0.01465397622087039, "context": 0.2781587343197316}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 3.6403918266296387, "loss_scale": 4.0, "retries": 0, "seconds": 155.7, "max_memory_gb": 17.11, "ce": 0.1434949460599455, "kl": 0.0036192588613630505, "context": 0.1453045760281384}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.30282336473464966, "loss_scale": 4.0, "retries": 0, "seconds": 156.8, "max_memory_gb": 17.11, "ce": 0.000865982445247937, "kl": 0.037138439442969684, "context": 0.019435202222666703}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 3.9081246852874756, "loss_scale": 4.0, "retries": 0, "seconds": 157.8, "max_memory_gb": 17.11, "ce": 0.832381353888195, "kl": 0.024544934334699064, "context": 0.8446538163116202}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 1.6778721809387207, "loss_scale": 4.0, "retries": 0, "seconds": 158.8, "max_memory_gb": 17.11, "ce": 0.010470362205523998, "kl": 0.3442655500257388, "context": 0.18260313104838133}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 1.1228936910629272, "loss_scale": 4.0, "retries": 0, "seconds": 159.8, "max_memory_gb": 17.11, "ce": 0.15810360113391653, "kl": 0.08995773302012822, "context": 0.20308247073262464}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 2.6314210891723633, "loss_scale": 4.0, "retries": 0, "seconds": 160.8, "max_memory_gb": 17.11, "ce": 0.08767864806577563, "kl": 0.5779351208475418, "context": 0.3766462022031192}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.03926866501569748, "loss_scale": 4.0, "retries": 0, "seconds": 161.8, "max_memory_gb": 17.11, "ce": 0.002250547244329937, "kl": 0.0017627586501021142, "context": 0.003131926612695679}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 1.9119799137115479, "loss_scale": 4.0, "retries": 0, "seconds": 162.9, "max_memory_gb": 17.11, "ce": 0.055060415139450924, "kl": 0.03056718246125456, "context": 0.07034400810152874}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 0.5476827621459961, "loss_scale": 4.0, "retries": 0, "seconds": 163.9, "max_memory_gb": 17.11, "ce": 0.015902938503131736, "kl": 0.025071783505609346, "context": 0.028438830457162112}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.2826700508594513, "loss_scale": 4.0, "retries": 0, "seconds": 164.9, "max_memory_gb": 17.11, "ce": 0.014897288550855592, "kl": 0.010639672487741336, "context": 0.02021712437272072}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 1.996079444885254, "loss_scale": 4.0, "retries": 0, "seconds": 166.0, "max_memory_gb": 17.11, "ce": 0.24922622663143557, "kl": 0.3890026982408017, "context": 0.4437275786040118}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 0.08637081831693649, "loss_scale": 4.0, "retries": 0, "seconds": 167.0, "max_memory_gb": 17.11, "ce": 0.0028569125206558965, "kl": 0.009864798672424513, "context": 0.0077893116358609404}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.014725583605468273, "loss_scale": 4.0, "retries": 0, "seconds": 168.1, "max_memory_gb": 17.11, "ce": 0.0009994113788707182, "kl": 0.00018126596501133463, "context": 0.001090044344891794}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.9533193111419678, "loss_scale": 4.0, "retries": 0, "seconds": 169.1, "max_memory_gb": 17.11, "ce": 0.002634499738633167, "kl": 0.38731105608167127, "context": 0.19629002179135568}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 0.6824740171432495, "loss_scale": 4.0, "retries": 0, "seconds": 170.1, "max_memory_gb": 17.11, "ce": 0.06459583772812039, "kl": 0.12399914497473219, "context": 0.1265954102273099}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 0.35927698016166687, "loss_scale": 4.0, "retries": 0, "seconds": 171.1, "max_memory_gb": 17.11, "ce": 0.014885911135934293, "kl": 0.04627396006253548, "context": 0.03802289126906544}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 1.2143083810806274, "loss_scale": 4.0, "retries": 0, "seconds": 172.1, "max_memory_gb": 17.11, "ce": 0.028074787027435377, "kl": 0.08810121174428787, "context": 0.0721253931405954}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 0.4181549847126007, "loss_scale": 4.0, "retries": 0, "seconds": 173.1, "max_memory_gb": 17.11, "ce": 0.02702305151615292, "kl": 0.01901630258066689, "context": 0.036531203251797706}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.030108951032161713, "loss_scale": 4.0, "retries": 0, "seconds": 174.2, "max_memory_gb": 17.11, "ce": 0.0015123698103707284, "kl": 0.001303631521295756, "context": 0.002164185541914776}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 0.0747770443558693, "loss_scale": 4.0, "retries": 0, "seconds": 175.2, "max_memory_gb": 17.11, "ce": 0.004990170389646664, "kl": 0.0002777108788905025, "context": 0.005129025856149383}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 1.521263837814331, "loss_scale": 4.0, "retries": 0, "seconds": 176.3, "max_memory_gb": 17.11, "ce": 0.30895693054480944, "kl": 0.00992114005930489, "context": 0.31391750255716033}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 1.8260308504104614, "loss_scale": 4.0, "retries": 0, "seconds": 177.3, "max_memory_gb": 17.11, "ce": 0.13939902518177405, "kl": 0.1731600423809141, "context": 0.22597904782742262}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 0.06358665972948074, "loss_scale": 4.0, "retries": 0, "seconds": 178.3, "max_memory_gb": 17.11, "ce": 0.0033950085926335305, "kl": 0.0008287830132758245, "context": 0.00380940007744357}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 1.1987032890319824, "loss_scale": 4.0, "retries": 0, "seconds": 179.4, "max_memory_gb": 17.11, "ce": 0.10931711381999776, "kl": 0.0017222154201590456, "context": 0.11017822113353759}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 1.465627670288086, "loss_scale": 4.0, "retries": 0, "seconds": 180.4, "max_memory_gb": 17.11, "ce": 0.06342698258231394, "kl": 0.08358407156629255, "context": 0.10521901835454628}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 2.9988465309143066, "loss_scale": 4.0, "retries": 0, "seconds": 181.3, "max_memory_gb": 17.11, "ce": 0.2192292729159817, "kl": 0.05571820749901235, "context": 0.24708838062360883}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.43861353397369385, "loss_scale": 4.0, "retries": 0, "seconds": 182.5, "max_memory_gb": 17.11, "ce": 0.03739025877439417, "kl": 0.030952608159395822, "context": 0.05286656350654084}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 3.747431755065918, "loss_scale": 4.0, "retries": 0, "seconds": 183.5, "max_memory_gb": 17.11, "ce": 0.26024639298339025, "kl": 0.010964189621290643, "context": 0.2657284887645801}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 1.5077675580978394, "loss_scale": 4.0, "retries": 0, "seconds": 184.4, "max_memory_gb": 17.11, "ce": 0.15541742992354557, "kl": 0.3641083729453385, "context": 0.3374716266989708}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.7837473154067993, "loss_scale": 4.0, "retries": 0, "seconds": 185.5, "max_memory_gb": 17.11, "ce": 0.13998883264139295, "kl": 0.036114986665779725, "context": 0.15804632741492242}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 1.3943305015563965, "loss_scale": 4.0, "retries": 0, "seconds": 186.5, "max_memory_gb": 17.11, "ce": 0.09484270142274909, "kl": 0.09272062417585403, "context": 0.14120301348157227}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 0.16000069677829742, "loss_scale": 4.0, "retries": 0, "seconds": 187.6, "max_memory_gb": 17.11, "ce": 0.011696719004248735, "kl": 0.0036343496662993857, "context": 0.013513893391063903}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 0.4565269649028778, "loss_scale": 4.0, "retries": 0, "seconds": 188.6, "max_memory_gb": 17.11, "ce": 0.06991602876223624, "kl": 0.013603215831608395, "context": 0.0767176382796606}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 2.948179006576538, "loss_scale": 4.0, "retries": 0, "seconds": 189.6, "max_memory_gb": 17.11, "ce": 0.009246833244105801, "kl": 0.15687919062793299, "context": 0.08768642986251507}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 1.4110667705535889, "loss_scale": 4.0, "retries": 0, "seconds": 190.7, "max_memory_gb": 17.11, "ce": 0.11161139828618616, "kl": 0.02811325725633651, "context": 0.12566802720539272}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.613699734210968, "loss_scale": 4.0, "retries": 0, "seconds": 191.7, "max_memory_gb": 17.11, "ce": 0.0025442448513786076, "kl": 0.09110591900571308, "context": 0.048097205959493294}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.5892589092254639, "loss_scale": 4.0, "retries": 0, "seconds": 192.7, "max_memory_gb": 17.11, "ce": 0.003278164833318442, "kl": 0.16986853578782757, "context": 0.0882124294876121}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 1.782759428024292, "loss_scale": 4.0, "retries": 0, "seconds": 193.8, "max_memory_gb": 17.11, "ce": 0.19547152105951682, "kl": 0.04623160371556878, "context": 0.218587328447029}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 3.222137689590454, "loss_scale": 4.0, "retries": 0, "seconds": 194.8, "max_memory_gb": 17.11, "ce": 0.11917981706210412, "kl": 0.021932613917670096, "context": 0.1301461224720697}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 1.8989613056182861, "loss_scale": 4.0, "retries": 0, "seconds": 195.8, "max_memory_gb": 17.11, "ce": 0.006831247737864032, "kl": 0.14591587703671394, "context": 0.07978918810840696}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 1.1085706949234009, "loss_scale": 4.0, "retries": 0, "seconds": 196.9, "max_memory_gb": 17.11, "ce": 0.10564784714370035, "kl": 0.2102426252895384, "context": 0.21076915420417208}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 0.6187416315078735, "loss_scale": 4.0, "retries": 0, "seconds": 197.9, "max_memory_gb": 17.11, "ce": 0.023120720026781783, "kl": 0.05023327941853495, "context": 0.04823735950049013}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 0.484199583530426, "loss_scale": 4.0, "retries": 0, "seconds": 198.9, "max_memory_gb": 17.11, "ce": 0.0033201585392816924, "kl": 0.12509157337353827, "context": 0.06586594432155835}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 9.552167892456055, "loss_scale": 4.0, "retries": 0, "seconds": 200.0, "max_memory_gb": 17.11, "ce": 0.17726319214852992, "kl": 0.19218445266596973, "context": 0.27335541865613777}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 0.043199654668569565, "loss_scale": 4.0, "retries": 0, "seconds": 201.0, "max_memory_gb": 17.11, "ce": 0.003346397541463375, "kl": 0.0008844414828672598, "context": 0.0037886183708906174}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.460201382637024, "loss_scale": 4.0, "retries": 0, "seconds": 202.1, "max_memory_gb": 17.11, "ce": 0.0091465238656383, "kl": 0.1146460891723109, "context": 0.06646956875920296}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.6100159287452698, "loss_scale": 4.0, "retries": 0, "seconds": 203.2, "max_memory_gb": 17.11, "ce": 0.0076702203004970215, "kl": 0.04044086213616538, "context": 0.02789065134129487}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 1.2723629474639893, "loss_scale": 4.0, "retries": 0, "seconds": 204.2, "max_memory_gb": 17.11, "ce": 0.04185177740873769, "kl": 0.16955409353977302, "context": 0.1266288251208607}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 1.1444218158721924, "loss_scale": 4.0, "retries": 0, "seconds": 205.3, "max_memory_gb": 17.11, "ce": 0.02874018714283011, "kl": 0.1565618616214124, "context": 0.10702112190483604}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 0.02795020304620266, "loss_scale": 4.0, "retries": 0, "seconds": 206.3, "max_memory_gb": 17.11, "ce": 0.001849292661063373, "kl": 0.0012244370243479352, "context": 0.0024615111760795116}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 0.01748545467853546, "loss_scale": 4.0, "retries": 0, "seconds": 207.3, "max_memory_gb": 17.11, "ce": 0.0007821651743142866, "kl": 0.00023115983776733628, "context": 0.0008977450925158337}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 1.204532504081726, "loss_scale": 4.0, "retries": 0, "seconds": 208.4, "max_memory_gb": 17.11, "ce": 0.048786393454065546, "kl": 0.3074617799547923, "context": 0.20251728681614622}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 0.02731163427233696, "loss_scale": 4.0, "retries": 0, "seconds": 209.4, "max_memory_gb": 17.11, "ce": 0.001664688010350801, "kl": 0.00018805268564392463, "context": 0.0017587143302080221}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.9920604228973389, "loss_scale": 4.0, "retries": 0, "seconds": 210.5, "max_memory_gb": 17.11, "ce": 0.01110170700121671, "kl": 0.843035900266841, "context": 0.432619655970484}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 0.4446868598461151, "loss_scale": 4.0, "retries": 0, "seconds": 211.6, "max_memory_gb": 17.11, "ce": 0.0387715813194518, "kl": 0.020907916230498813, "context": 0.049225537484744564}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.26926037669181824, "loss_scale": 4.0, "retries": 0, "seconds": 212.6, "max_memory_gb": 17.11, "ce": 0.01829980291586253, "kl": 0.004501964974224393, "context": 0.020550784960505553}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 1.7968977689743042, "loss_scale": 4.0, "retries": 0, "seconds": 213.7, "max_memory_gb": 17.11, "ce": 0.060946262994548306, "kl": 0.02101037377724424, "context": 0.0714514511055313}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.049927085638046265, "loss_scale": 4.0, "retries": 0, "seconds": 214.7, "max_memory_gb": 17.11, "ce": 0.0034596762561704963, "kl": 0.0002704121397982817, "context": 0.003594882320612669}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 2.54580020904541, "loss_scale": 4.0, "retries": 0, "seconds": 215.7, "max_memory_gb": 17.11, "ce": 0.326066081848694, "kl": 0.007454689315636642, "context": 0.32979343548504403}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 1.4834142923355103, "loss_scale": 4.0, "retries": 0, "seconds": 216.8, "max_memory_gb": 17.11, "ce": 0.18318058125441894, "kl": 0.5305312852688076, "context": 0.44844622374512255}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.15213528275489807, "loss_scale": 4.0, "retries": 0, "seconds": 217.8, "max_memory_gb": 17.11, "ce": 0.006977995777560864, "kl": 0.0017924794398140875, "context": 0.007874235387134831}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.12486927956342697, "loss_scale": 4.0, "retries": 0, "seconds": 218.8, "max_memory_gb": 17.11, "ce": 0.012014799678581767, "kl": 0.0015419876376654429, "context": 0.01278579379140865}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 0.051357027143239975, "loss_scale": 4.0, "retries": 0, "seconds": 219.9, "max_memory_gb": 17.11, "ce": 0.004160250711720437, "kl": 0.0017490832542534918, "context": 0.005034792353399098}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 2.9081568717956543, "loss_scale": 4.0, "retries": 0, "seconds": 220.9, "max_memory_gb": 17.11, "ce": 0.039793868469132576, "kl": 0.007420725945848972, "context": 0.0435042309545679}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 3.0543341636657715, "loss_scale": 4.0, "retries": 0, "seconds": 222.0, "max_memory_gb": 17.11, "ce": 0.63848941498145, "kl": 0.19615150173194706, "context": 0.7365651341970079}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 0.8316230773925781, "loss_scale": 4.0, "retries": 0, "seconds": 223.0, "max_memory_gb": 17.11, "ce": 0.03935203355649719, "kl": 0.07621218929034512, "context": 0.07745812850771472}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.8497313857078552, "loss_scale": 4.0, "retries": 0, "seconds": 224.0, "max_memory_gb": 17.11, "ce": 0.07261269248556346, "kl": 0.05122472172661219, "context": 0.09822505380725488}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 0.6268611550331116, "loss_scale": 4.0, "retries": 0, "seconds": 225.1, "max_memory_gb": 17.11, "ce": 0.07584533852059394, "kl": 0.04233922724961303, "context": 0.09701495035551488}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 0.3659816086292267, "loss_scale": 4.0, "retries": 0, "seconds": 226.1, "max_memory_gb": 17.11, "ce": 0.012984223722014576, "kl": 0.006340786972941714, "context": 0.01615461708570365}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.3276192843914032, "loss_scale": 4.0, "retries": 0, "seconds": 227.1, "max_memory_gb": 17.11, "ce": 0.009282572355004959, "kl": 0.09353150083984474, "context": 0.05604832207609434}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.2297709733247757, "loss_scale": 4.0, "retries": 0, "seconds": 228.3, "max_memory_gb": 17.11, "ce": 0.020225850661518052, "kl": 0.004125070074223913, "context": 0.02228838563314639}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 0.781366765499115, "loss_scale": 4.0, "retries": 0, "seconds": 229.3, "max_memory_gb": 17.11, "ce": 0.10060866071944474, "kl": 0.037731450929186394, "context": 0.11947438434071955}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 0.8227874040603638, "loss_scale": 4.0, "retries": 0, "seconds": 230.3, "max_memory_gb": 17.11, "ce": 0.050651432073209435, "kl": 0.001030513652949594, "context": 0.051166690624086186}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 1.0762439966201782, "loss_scale": 4.0, "retries": 0, "seconds": 231.4, "max_memory_gb": 17.11, "ce": 0.03846273848466808, "kl": 0.2673373629077105, "context": 0.17213142357650213}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.4664524793624878, "loss_scale": 4.0, "retries": 0, "seconds": 232.4, "max_memory_gb": 17.11, "ce": 0.02062781050335616, "kl": 0.007893480894836102, "context": 0.02457455001422204}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.9443016052246094, "loss_scale": 4.0, "retries": 0, "seconds": 233.4, "max_memory_gb": 17.11, "ce": 0.08963943022536114, "kl": 0.06116904558348324, "context": 0.120223955163965}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 0.9102480411529541, "loss_scale": 4.0, "retries": 0, "seconds": 234.5, "max_memory_gb": 17.11, "ce": 0.00834409796516411, "kl": 0.3890935630745389, "context": 0.20289088346180506}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 1.0386383533477783, "loss_scale": 4.0, "retries": 0, "seconds": 235.5, "max_memory_gb": 17.11, "ce": 0.03946374605584424, "kl": 0.06663845967727866, "context": 0.07278297588345595}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.8876461982727051, "loss_scale": 4.0, "retries": 0, "seconds": 236.6, "max_memory_gb": 17.11, "ce": 0.060561607526324224, "kl": 0.08253096888574873, "context": 0.10182709282526048}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.7256231307983398, "loss_scale": 4.0, "retries": 0, "seconds": 237.6, "max_memory_gb": 17.11, "ce": 0.04546277846384328, "kl": 0.020637223999074195, "context": 0.05578138882992789}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 1.5937029123306274, "loss_scale": 4.0, "retries": 0, "seconds": 238.6, "max_memory_gb": 17.11, "ce": 0.012764021230395883, "kl": 0.19889289693674073, "context": 0.11221046553691849}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 2.5131750106811523, "loss_scale": 4.0, "retries": 0, "seconds": 239.7, "max_memory_gb": 17.11, "ce": 0.041491206455248175, "kl": 0.5343413488149054, "context": 0.30866187474748585}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 0.030790148302912712, "loss_scale": 4.0, "retries": 0, "seconds": 240.7, "max_memory_gb": 17.11, "ce": 0.002262623187561985, "kl": 0.0006441891782742459, "context": 0.0025847178185358644}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 0.4279383420944214, "loss_scale": 4.0, "retries": 0, "seconds": 241.8, "max_memory_gb": 17.11, "ce": 0.011625829734839499, "kl": 0.015548060597211588, "context": 0.01939986099023372}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 2.1079699993133545, "loss_scale": 4.0, "retries": 0, "seconds": 242.9, "max_memory_gb": 17.11, "ce": 0.08032118595656357, "kl": 0.25706144416471943, "context": 0.20885190879926085}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.026760218665003777, "loss_scale": 4.0, "retries": 0, "seconds": 243.9, "max_memory_gb": 17.11, "ce": 0.001752738084178418, "kl": 0.0003569092939414986, "context": 0.0019311926953378133}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 1.1109185218811035, "loss_scale": 4.0, "retries": 0, "seconds": 245.0, "max_memory_gb": 17.11, "ce": 0.10499958071159199, "kl": 0.05669175253933645, "context": 0.1333454608102329}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 2.925469398498535, "loss_scale": 4.0, "retries": 0, "seconds": 246.0, "max_memory_gb": 17.11, "ce": 0.06620323131937766, "kl": 0.4025448460597545, "context": 0.26747564668767154}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 2.7155730724334717, "loss_scale": 4.0, "retries": 0, "seconds": 247.0, "max_memory_gb": 17.11, "ce": 0.8900643125889474, "kl": 0.021644728709361516, "context": 0.9008866506774211}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.11377188563346863, "loss_scale": 4.0, "retries": 0, "seconds": 248.1, "max_memory_gb": 17.11, "ce": 0.007786409420077689, "kl": 0.005401909238571534, "context": 0.010487364022992551}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 1.20657479763031, "loss_scale": 4.0, "retries": 0, "seconds": 249.1, "max_memory_gb": 17.11, "ce": 0.021019665524363518, "kl": 0.002266652508296829, "context": 0.022152991077746265}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.16189676523208618, "loss_scale": 4.0, "retries": 0, "seconds": 250.1, "max_memory_gb": 17.11, "ce": 0.0021715863840654492, "kl": 0.01815192536196264, "context": 0.01124754938064143}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 1.3084214925765991, "loss_scale": 4.0, "retries": 0, "seconds": 251.2, "max_memory_gb": 17.11, "ce": 0.020769201189978048, "kl": 0.4577328715895419, "context": 0.2496356320916675}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 1.1418129205703735, "loss_scale": 4.0, "retries": 0, "seconds": 252.2, "max_memory_gb": 17.11, "ce": 0.03139438733342104, "kl": 0.03145822864098591, "context": 0.04712350372574292}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 1.0979028940200806, "loss_scale": 4.0, "retries": 0, "seconds": 253.2, "max_memory_gb": 17.11, "ce": 0.1327444739290513, "kl": 0.046564286338252714, "context": 0.15602661337470636}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.3881258964538574, "loss_scale": 4.0, "retries": 0, "seconds": 254.3, "max_memory_gb": 17.11, "ce": 0.03796098122256808, "kl": 0.010222043871181086, "context": 0.04307200270704925}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 1.1177339553833008, "loss_scale": 4.0, "retries": 0, "seconds": 255.3, "max_memory_gb": 17.11, "ce": 0.1348329170141369, "kl": 0.022212653508177027, "context": 0.14593924663495272}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 3.1918203830718994, "loss_scale": 4.0, "retries": 0, "seconds": 256.4, "max_memory_gb": 17.11, "ce": 0.1074590444623027, "kl": 0.07205651133881474, "context": 0.14348729641642421}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 0.34084832668304443, "loss_scale": 4.0, "retries": 0, "seconds": 257.4, "max_memory_gb": 17.11, "ce": 0.026794745499501005, "kl": 0.005506223537395272, "context": 0.02954785749898292}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.9166127443313599, "loss_scale": 4.0, "retries": 0, "seconds": 258.3, "max_memory_gb": 17.11, "ce": 0.166147728334181, "kl": 0.011632640002062544, "context": 0.171964043576736}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 4.6906280517578125, "loss_scale": 4.0, "retries": 0, "seconds": 259.4, "max_memory_gb": 17.11, "ce": 0.0851706286193803, "kl": 0.11485451592307072, "context": 0.1425978884799406}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.042288683354854584, "loss_scale": 4.0, "retries": 0, "seconds": 260.4, "max_memory_gb": 17.11, "ce": 0.001950180194398854, "kl": 0.002014227655990908, "context": 0.0029572940402431414}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 0.32259899377822876, "loss_scale": 4.0, "retries": 0, "seconds": 261.4, "max_memory_gb": 17.11, "ce": 0.02254656289005652, "kl": 0.020392237882333575, "context": 0.03274268298991956}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 1.4893128871917725, "loss_scale": 4.0, "retries": 0, "seconds": 262.5, "max_memory_gb": 17.11, "ce": 0.03724514544592239, "kl": 0.03749893247731961, "context": 0.05599461239762604}

runs/fixed_internvl/context/Random8_8101_s13/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Random8_8101_s13/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.1.mlp.down_proj": 144,
+    "model.language_model.layers.1.mlp.gate_proj": 144,
+    "model.language_model.layers.1.mlp.up_proj": 144,
+    "model.language_model.layers.1.self_attn.k_proj": 144,
+    "model.language_model.layers.1.self_attn.o_proj": 144,
+    "model.language_model.layers.1.self_attn.q_proj": 144,
+    "model.language_model.layers.1.self_attn.v_proj": 144,
+    "model.language_model.layers.10.mlp.down_proj": 144,
+    "model.language_model.layers.10.mlp.gate_proj": 144,
+    "model.language_model.layers.10.mlp.up_proj": 144,
+    "model.language_model.layers.10.self_attn.k_proj": 144,
+    "model.language_model.layers.10.self_attn.o_proj": 144,
+    "model.language_model.layers.10.self_attn.q_proj": 144,
+    "model.language_model.layers.10.self_attn.v_proj": 144,
+    "model.language_model.layers.21.mlp.down_proj": 144,
+    "model.language_model.layers.21.mlp.gate_proj": 144,
+    "model.language_model.layers.21.mlp.up_proj": 144,
+    "model.language_model.layers.21.self_attn.k_proj": 144,
+    "model.language_model.layers.21.self_attn.o_proj": 144,
+    "model.language_model.layers.21.self_attn.q_proj": 144,
+    "model.language_model.layers.21.self_attn.v_proj": 144,
+    "model.language_model.layers.28.mlp.down_proj": 144,
+    "model.language_model.layers.28.mlp.gate_proj": 144,
+    "model.language_model.layers.28.mlp.up_proj": 144,
+    "model.language_model.layers.28.self_attn.k_proj": 144,
+    "model.language_model.layers.28.self_attn.o_proj": 144,
+    "model.language_model.layers.28.self_attn.q_proj": 144,
+    "model.language_model.layers.28.self_attn.v_proj": 144,
+    "model.language_model.layers.31.mlp.down_proj": 144,
+    "model.language_model.layers.31.mlp.gate_proj": 144,
+    "model.language_model.layers.31.mlp.up_proj": 144,
+    "model.language_model.layers.31.self_attn.k_proj": 144,
+    "model.language_model.layers.31.self_attn.o_proj": 144,
+    "model.language_model.layers.31.self_attn.q_proj": 144,
+    "model.language_model.layers.31.self_attn.v_proj": 144,
+    "model.language_model.layers.35.mlp.down_proj": 144,
+    "model.language_model.layers.35.mlp.gate_proj": 144,
+    "model.language_model.layers.35.mlp.up_proj": 144,
+    "model.language_model.layers.35.self_attn.k_proj": 144,
+    "model.language_model.layers.35.self_attn.o_proj": 144,
+    "model.language_model.layers.35.self_attn.q_proj": 144,
+    "model.language_model.layers.35.self_attn.v_proj": 144,
+    "model.language_model.layers.4.mlp.down_proj": 144,
+    "model.language_model.layers.4.mlp.gate_proj": 144,
+    "model.language_model.layers.4.mlp.up_proj": 144,
+    "model.language_model.layers.4.self_attn.k_proj": 144,
+    "model.language_model.layers.4.self_attn.o_proj": 144,
+    "model.language_model.layers.4.self_attn.q_proj": 144,
+    "model.language_model.layers.4.self_attn.v_proj": 144,
+    "model.language_model.layers.9.mlp.down_proj": 144,
+    "model.language_model.layers.9.mlp.gate_proj": 144,
+    "model.language_model.layers.9.mlp.up_proj": 144,
+    "model.language_model.layers.9.self_attn.k_proj": 144,
+    "model.language_model.layers.9.self_attn.o_proj": 144,
+    "model.language_model.layers.9.self_attn.q_proj": 144,
+    "model.language_model.layers.9.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.1.mlp.down_proj": 72,
+    "model.language_model.layers.1.mlp.gate_proj": 72,
+    "model.language_model.layers.1.mlp.up_proj": 72,
+    "model.language_model.layers.1.self_attn.k_proj": 72,
+    "model.language_model.layers.1.self_attn.o_proj": 72,
+    "model.language_model.layers.1.self_attn.q_proj": 72,
+    "model.language_model.layers.1.self_attn.v_proj": 72,
+    "model.language_model.layers.10.mlp.down_proj": 72,
+    "model.language_model.layers.10.mlp.gate_proj": 72,
+    "model.language_model.layers.10.mlp.up_proj": 72,
+    "model.language_model.layers.10.self_attn.k_proj": 72,
+    "model.language_model.layers.10.self_attn.o_proj": 72,
+    "model.language_model.layers.10.self_attn.q_proj": 72,
+    "model.language_model.layers.10.self_attn.v_proj": 72,
+    "model.language_model.layers.21.mlp.down_proj": 72,
+    "model.language_model.layers.21.mlp.gate_proj": 72,
+    "model.language_model.layers.21.mlp.up_proj": 72,
+    "model.language_model.layers.21.self_attn.k_proj": 72,
+    "model.language_model.layers.21.self_attn.o_proj": 72,
+    "model.language_model.layers.21.self_attn.q_proj": 72,
+    "model.language_model.layers.21.self_attn.v_proj": 72,
+    "model.language_model.layers.28.mlp.down_proj": 72,
+    "model.language_model.layers.28.mlp.gate_proj": 72,
+    "model.language_model.layers.28.mlp.up_proj": 72,
+    "model.language_model.layers.28.self_attn.k_proj": 72,
+    "model.language_model.layers.28.self_attn.o_proj": 72,
+    "model.language_model.layers.28.self_attn.q_proj": 72,
+    "model.language_model.layers.28.self_attn.v_proj": 72,
+    "model.language_model.layers.31.mlp.down_proj": 72,
+    "model.language_model.layers.31.mlp.gate_proj": 72,
+    "model.language_model.layers.31.mlp.up_proj": 72,
+    "model.language_model.layers.31.self_attn.k_proj": 72,
+    "model.language_model.layers.31.self_attn.o_proj": 72,
+    "model.language_model.layers.31.self_attn.q_proj": 72,
+    "model.language_model.layers.31.self_attn.v_proj": 72,
+    "model.language_model.layers.35.mlp.down_proj": 72,
+    "model.language_model.layers.35.mlp.gate_proj": 72,
+    "model.language_model.layers.35.mlp.up_proj": 72,
+    "model.language_model.layers.35.self_attn.k_proj": 72,
+    "model.language_model.layers.35.self_attn.o_proj": 72,
+    "model.language_model.layers.35.self_attn.q_proj": 72,
+    "model.language_model.layers.35.self_attn.v_proj": 72,
+    "model.language_model.layers.4.mlp.down_proj": 72,
+    "model.language_model.layers.4.mlp.gate_proj": 72,
+    "model.language_model.layers.4.mlp.up_proj": 72,
+    "model.language_model.layers.4.self_attn.k_proj": 72,
+    "model.language_model.layers.4.self_attn.o_proj": 72,
+    "model.language_model.layers.4.self_attn.q_proj": 72,
+    "model.language_model.layers.4.self_attn.v_proj": 72,
+    "model.language_model.layers.9.mlp.down_proj": 72,
+    "model.language_model.layers.9.mlp.gate_proj": 72,
+    "model.language_model.layers.9.mlp.up_proj": 72,
+    "model.language_model.layers.9.self_attn.k_proj": 72,
+    "model.language_model.layers.9.self_attn.o_proj": 72,
+    "model.language_model.layers.9.self_attn.q_proj": 72,
+    "model.language_model.layers.9.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.4.mlp.gate_proj",
+    "model.language_model.layers.10.mlp.up_proj",
+    "model.language_model.layers.31.self_attn.v_proj",
+    "model.language_model.layers.35.mlp.down_proj",
+    "model.language_model.layers.9.self_attn.o_proj",
+    "model.language_model.layers.10.self_attn.v_proj",
+    "model.language_model.layers.35.mlp.gate_proj",
+    "model.language_model.layers.4.self_attn.v_proj",
+    "model.language_model.layers.31.mlp.down_proj",
+    "model.language_model.layers.9.mlp.down_proj",
+    "model.language_model.layers.1.self_attn.o_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.1.mlp.gate_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.4.mlp.up_proj",
+    "model.language_model.layers.9.self_attn.q_proj",
+    "model.language_model.layers.4.self_attn.k_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.10.self_attn.k_proj",
+    "model.language_model.layers.31.self_attn.k_proj",
+    "model.language_model.layers.28.self_attn.v_proj",
+    "model.language_model.layers.1.self_attn.q_proj",
+    "model.language_model.layers.9.mlp.up_proj",
+    "model.language_model.layers.4.self_attn.o_proj",
+    "model.language_model.layers.4.self_attn.q_proj",
+    "model.language_model.layers.31.mlp.gate_proj",
+    "model.language_model.layers.35.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.28.self_attn.k_proj",
+    "model.language_model.layers.31.self_attn.o_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.10.self_attn.q_proj",
+    "model.language_model.layers.1.mlp.down_proj",
+    "model.language_model.layers.35.mlp.up_proj",
+    "model.language_model.layers.10.mlp.gate_proj",
+    "model.language_model.layers.10.mlp.down_proj",
+    "model.language_model.layers.35.self_attn.v_proj",
+    "model.language_model.layers.9.mlp.gate_proj",
+    "model.language_model.layers.1.self_attn.v_proj",
+    "model.language_model.layers.10.self_attn.o_proj",
+    "model.language_model.layers.31.self_attn.q_proj",
+    "model.language_model.layers.4.mlp.down_proj",
+    "model.language_model.layers.1.self_attn.k_proj",
+    "model.language_model.layers.35.self_attn.o_proj",
+    "model.language_model.layers.1.mlp.up_proj",
+    "model.language_model.layers.35.self_attn.k_proj",
+    "model.language_model.layers.9.self_attn.k_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.31.mlp.up_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.v_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.28.mlp.gate_proj",
+    "model.language_model.layers.28.mlp.down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Random8_8101_s13/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 13,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Random8_8101",
+    "layers": [
+      1,
+      4,
+      9,
+      10,
+      21,
+      28,
+      31,
+      35
+    ],
+    "ranks": {
+      "1": 72,
+      "4": 72,
+      "9": 72,
+      "10": 72,
+      "21": 72,
+      "28": 72,
+      "31": 72,
+      "35": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Random8_8101_s13/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 4.357049942016602, "loss_scale": 4.0, "retries": 0, "seconds": 2.1, "max_memory_gb": 16.82, "ce": 0.5227766726748087, "kl": 0.08654405153356493, "context": 0.5660487064160407}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 5.944469928741455, "loss_scale": 4.0, "retries": 0, "seconds": 3.5, "max_memory_gb": 17.16, "ce": 0.6351806127713644, "kl": 0.46619299386225066, "context": 0.8682771419480559}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 16.09759521484375, "loss_scale": 4.0, "retries": 0, "seconds": 4.6, "max_memory_gb": 17.16, "ce": 1.0133285820484161, "kl": 0.0708104856312275, "context": 1.0487338304519653}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 6.590793609619141, "loss_scale": 4.0, "retries": 0, "seconds": 5.9, "max_memory_gb": 17.16, "ce": 0.31668926227575867, "kl": 0.44615664927368925, "context": 0.5397675840576994}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 0.17476053535938263, "loss_scale": 4.0, "retries": 0, "seconds": 7.1, "max_memory_gb": 17.17, "ce": 0.009469470092881238, "kl": 0.008445698265859392, "context": 0.01369231963326456}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 3.1055104732513428, "loss_scale": 4.0, "retries": 0, "seconds": 8.3, "max_memory_gb": 17.17, "ce": 0.17383333183533978, "kl": 0.07938875182662741, "context": 0.21352770797966514}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 4.744941711425781, "loss_scale": 4.0, "retries": 0, "seconds": 9.6, "max_memory_gb": 17.17, "ce": 0.18488641532530892, "kl": 0.08130240592959126, "context": 0.2255376229459216}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 2.773805856704712, "loss_scale": 4.0, "retries": 0, "seconds": 10.8, "max_memory_gb": 17.17, "ce": 0.08660546503961086, "kl": 0.21807282115332782, "context": 0.19564186967909336}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 5.619935989379883, "loss_scale": 4.0, "retries": 0, "seconds": 11.9, "max_memory_gb": 17.17, "ce": 0.10291691776365042, "kl": 0.156916132371407, "context": 0.181374984793365}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 0.6289013624191284, "loss_scale": 4.0, "retries": 0, "seconds": 13.2, "max_memory_gb": 17.17, "ce": 0.006065656758437399, "kl": 0.024582151789218187, "context": 0.018356732791289687}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.23286566138267517, "loss_scale": 4.0, "retries": 0, "seconds": 14.4, "max_memory_gb": 17.17, "ce": 0.00876860514836153, "kl": 0.004345735366769077, "context": 0.010941472944978159}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 0.22964364290237427, "loss_scale": 4.0, "retries": 0, "seconds": 15.6, "max_memory_gb": 17.17, "ce": 0.011375440683877969, "kl": 0.0021376972345024114, "context": 0.012444289532140829}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.0840483158826828, "loss_scale": 4.0, "retries": 0, "seconds": 16.8, "max_memory_gb": 17.17, "ce": 0.0024271288039017236, "kl": 0.00079356639253092, "context": 0.0028239120438229293}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 2.8395328521728516, "loss_scale": 4.0, "retries": 0, "seconds": 18.0, "max_memory_gb": 17.17, "ce": 0.0768673017155379, "kl": 0.7245392926270142, "context": 0.4391369433142245}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 1.3569762706756592, "loss_scale": 4.0, "retries": 0, "seconds": 19.3, "max_memory_gb": 17.17, "ce": 0.007370180108637214, "kl": 0.3417649542137724, "context": 0.17825265660439982}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 0.04146622493863106, "loss_scale": 4.0, "retries": 0, "seconds": 20.5, "max_memory_gb": 17.17, "ce": 0.002368376728554722, "kl": 0.0008819719000712212, "context": 0.002809362798871007}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 1.7308766841888428, "loss_scale": 4.0, "retries": 0, "seconds": 21.7, "max_memory_gb": 17.17, "ce": 0.034454015927622095, "kl": 0.8038506430966663, "context": 0.43637934655998833}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 1.0194162130355835, "loss_scale": 4.0, "retries": 0, "seconds": 23.0, "max_memory_gb": 17.17, "ce": 0.006468934610893484, "kl": 0.2552712828764925, "context": 0.13410457235295326}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.4930908977985382, "loss_scale": 4.0, "retries": 0, "seconds": 24.1, "max_memory_gb": 17.17, "ce": 0.005295447388107277, "kl": 0.06627227261196822, "context": 0.038431582988778246}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.101788729429245, "loss_scale": 4.0, "retries": 0, "seconds": 25.3, "max_memory_gb": 17.17, "ce": 0.0009912862160490477, "kl": 0.007789777317157132, "context": 0.004886174821876921}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.02881629578769207, "loss_scale": 4.0, "retries": 0, "seconds": 26.6, "max_memory_gb": 17.17, "ce": 0.00042424215826031286, "kl": 0.0017971875276998617, "context": 0.0013228359257482225}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 1.8555070161819458, "loss_scale": 4.0, "retries": 0, "seconds": 27.8, "max_memory_gb": 17.17, "ce": 0.06785891023901058, "kl": 0.05294014408264047, "context": 0.09432898274826584}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 3.7011501789093018, "loss_scale": 4.0, "retries": 0, "seconds": 29.0, "max_memory_gb": 17.17, "ce": 0.000885026993273641, "kl": 0.33688031127144313, "context": 0.16932518291287124}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.5984573364257812, "loss_scale": 4.0, "retries": 0, "seconds": 30.3, "max_memory_gb": 17.17, "ce": 0.007797622922225855, "kl": 0.03368307545406424, "context": 0.024639161354571115}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 2.7640395164489746, "loss_scale": 4.0, "retries": 0, "seconds": 31.5, "max_memory_gb": 17.17, "ce": 0.13901632851593604, "kl": 0.030680537573061883, "context": 0.15435659396462142}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 1.3472920656204224, "loss_scale": 4.0, "retries": 0, "seconds": 32.7, "max_memory_gb": 17.17, "ce": 0.041387547746126074, "kl": 0.5462893977028216, "context": 0.31453224288634374}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 1.5748240947723389, "loss_scale": 4.0, "retries": 0, "seconds": 34.0, "max_memory_gb": 17.17, "ce": 0.07138236639730167, "kl": 0.26222897212574026, "context": 0.2024968520127004}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 2.7684011459350586, "loss_scale": 4.0, "retries": 0, "seconds": 35.2, "max_memory_gb": 17.17, "ce": 0.1385459097950843, "kl": 0.22292192454915494, "context": 0.25000686571002007}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 3.659517526626587, "loss_scale": 4.0, "retries": 0, "seconds": 36.5, "max_memory_gb": 17.17, "ce": 0.49292756855720654, "kl": 0.14668776543112472, "context": 0.5662714641075581}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.27677762508392334, "loss_scale": 4.0, "retries": 0, "seconds": 37.8, "max_memory_gb": 17.17, "ce": 0.0033131438849522965, "kl": 0.010737004024349517, "context": 0.008681646100740181}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 5.393149375915527, "loss_scale": 4.0, "retries": 0, "seconds": 39.1, "max_memory_gb": 17.17, "ce": 0.6159597974183271, "kl": 0.07108478923328221, "context": 0.6515021638479084}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 1.2682983875274658, "loss_scale": 4.0, "retries": 0, "seconds": 40.5, "max_memory_gb": 17.17, "ce": 0.028075316338799894, "kl": 0.034076928452122957, "context": 0.04511378228198737}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 3.034740924835205, "loss_scale": 4.0, "retries": 0, "seconds": 41.7, "max_memory_gb": 17.17, "ce": 0.30610482732299715, "kl": 0.06432532845064998, "context": 0.3382674807216972}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.5379753112792969, "loss_scale": 4.0, "retries": 0, "seconds": 43.0, "max_memory_gb": 17.17, "ce": 0.006431674351915717, "kl": 0.7745311438338831, "context": 0.39369725377764553}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.06706560403108597, "loss_scale": 4.0, "retries": 0, "seconds": 44.3, "max_memory_gb": 17.17, "ce": 0.0020745015663123922, "kl": 0.0004124968463656842, "context": 0.0022807500172348227}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 2.6414573192596436, "loss_scale": 4.0, "retries": 0, "seconds": 45.5, "max_memory_gb": 17.17, "ce": 0.2151014574628789, "kl": 0.032936606603470864, "context": 0.23156976682366803}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 2.5971343517303467, "loss_scale": 4.0, "retries": 0, "seconds": 46.8, "max_memory_gb": 17.17, "ce": 0.1006699344725348, "kl": 0.5026339497417212, "context": 0.3519869092851877}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 1.934492588043213, "loss_scale": 4.0, "retries": 0, "seconds": 48.0, "max_memory_gb": 17.17, "ce": 0.06418829018366523, "kl": 0.47192345416988246, "context": 0.3001500216778368}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.135955810546875, "loss_scale": 4.0, "retries": 0, "seconds": 49.1, "max_memory_gb": 17.17, "ce": 0.07644374962546863, "kl": 0.007150884088332532, "context": 0.08001919023809023}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.5604733228683472, "loss_scale": 4.0, "retries": 0, "seconds": 50.4, "max_memory_gb": 17.17, "ce": 0.01782746729440987, "kl": 0.03002111316891387, "context": 0.032838023733347654}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 2.8161630630493164, "loss_scale": 4.0, "retries": 0, "seconds": 51.6, "max_memory_gb": 17.17, "ce": 0.4709232528693974, "kl": 0.07921667397022247, "context": 0.510531585663557}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 0.12885095179080963, "loss_scale": 4.0, "retries": 0, "seconds": 52.7, "max_memory_gb": 17.17, "ce": 0.007769485499011353, "kl": 0.0007524175634898711, "context": 0.00814569415524602}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 0.09712743759155273, "loss_scale": 4.0, "retries": 0, "seconds": 54.1, "max_memory_gb": 17.17, "ce": 0.005092534760478884, "kl": 0.00073520723162801, "context": 0.0054601384326815605}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 9.061918258666992, "loss_scale": 4.0, "retries": 0, "seconds": 55.3, "max_memory_gb": 17.17, "ce": 0.15193569456459954, "kl": 0.8223653972090688, "context": 0.5631183669902384}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 3.6105797290802, "loss_scale": 4.0, "retries": 0, "seconds": 56.5, "max_memory_gb": 17.17, "ce": 0.4475208844523877, "kl": 0.19020725980226416, "context": 0.5426245201379061}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 1.0547245740890503, "loss_scale": 4.0, "retries": 0, "seconds": 57.8, "max_memory_gb": 17.17, "ce": 0.06713016134744976, "kl": 0.12989442818798125, "context": 0.1320773740299046}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 1.1802524328231812, "loss_scale": 4.0, "retries": 0, "seconds": 59.0, "max_memory_gb": 17.17, "ce": 0.037751280004158616, "kl": 0.1639151311101159, "context": 0.11970884550828487}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.06286527961492538, "loss_scale": 4.0, "retries": 0, "seconds": 60.3, "max_memory_gb": 17.17, "ce": 0.0027774831978604198, "kl": 0.003017180548340548, "context": 0.004286073497496545}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.31161928176879883, "loss_scale": 4.0, "retries": 0, "seconds": 61.5, "max_memory_gb": 17.17, "ce": 0.011350332279107533, "kl": 0.023776310314133298, "context": 0.02323848774540238}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 1.0185935497283936, "loss_scale": 4.0, "retries": 0, "seconds": 62.6, "max_memory_gb": 17.17, "ce": 0.0688658767612651, "kl": 0.01018772523093503, "context": 0.07395973996608518}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 2.8825955390930176, "loss_scale": 4.0, "retries": 0, "seconds": 63.9, "max_memory_gb": 17.17, "ce": 0.05869163782335818, "kl": 0.004467266286155791, "context": 0.060925272686290555}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.970278263092041, "loss_scale": 4.0, "retries": 0, "seconds": 65.0, "max_memory_gb": 17.17, "ce": 0.01801416277885437, "kl": 0.1253173159493599, "context": 0.08067281963303685}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 1.4541022777557373, "loss_scale": 4.0, "retries": 0, "seconds": 66.2, "max_memory_gb": 17.17, "ce": 0.03738933021668345, "kl": 0.20510728820227087, "context": 0.13994297722820193}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 2.3520169258117676, "loss_scale": 4.0, "retries": 0, "seconds": 67.5, "max_memory_gb": 17.17, "ce": 0.15534974556067027, "kl": 0.08286351434071548, "context": 0.19678150388062932}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 0.25427618622779846, "loss_scale": 4.0, "retries": 0, "seconds": 68.7, "max_memory_gb": 17.17, "ce": 0.01664741651620716, "kl": 0.006863765867819893, "context": 0.020079299807548523}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.5959200859069824, "loss_scale": 4.0, "retries": 0, "seconds": 69.9, "max_memory_gb": 17.17, "ce": 0.022945079836063087, "kl": 0.06994795467471704, "context": 0.05791905662044883}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.21366485953330994, "loss_scale": 4.0, "retries": 0, "seconds": 71.1, "max_memory_gb": 17.17, "ce": 0.008376234269235283, "kl": 0.010516137510421686, "context": 0.013634303526487201}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.03618164733052254, "loss_scale": 4.0, "retries": 0, "seconds": 72.3, "max_memory_gb": 17.17, "ce": 0.0018428936746204272, "kl": 0.0003324729477753863, "context": 0.0020091301121283323}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 2.00268816947937, "loss_scale": 4.0, "retries": 0, "seconds": 73.8, "max_memory_gb": 17.17, "ce": 0.173544007120654, "kl": 0.07655737301683985, "context": 0.21182269550627097}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 1.6579561233520508, "loss_scale": 4.0, "retries": 0, "seconds": 75.2, "max_memory_gb": 17.17, "ce": 0.1336465856875293, "kl": 0.09183004213264212, "context": 0.17956160311587155}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 1.395755410194397, "loss_scale": 4.0, "retries": 0, "seconds": 76.4, "max_memory_gb": 17.17, "ce": 0.09809538867557421, "kl": 0.4542900474771159, "context": 0.3252404183149338}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.5114976167678833, "loss_scale": 4.0, "retries": 0, "seconds": 77.7, "max_memory_gb": 17.17, "ce": 0.04648087051464245, "kl": 0.010589061363134533, "context": 0.05177540169097483}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 1.450404405593872, "loss_scale": 4.0, "retries": 0, "seconds": 78.9, "max_memory_gb": 17.17, "ce": 0.09553750467603095, "kl": 0.10543263745785225, "context": 0.14825381990522146}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.31591594219207764, "loss_scale": 4.0, "retries": 0, "seconds": 80.1, "max_memory_gb": 17.17, "ce": 0.018292045453563333, "kl": 0.008802592346910387, "context": 0.022693340899422765}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 0.807854175567627, "loss_scale": 4.0, "retries": 0, "seconds": 81.4, "max_memory_gb": 17.17, "ce": 0.12832721415907145, "kl": 0.11511379620060325, "context": 0.18588411062955856}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 2.7875678539276123, "loss_scale": 4.0, "retries": 0, "seconds": 82.6, "max_memory_gb": 17.17, "ce": 0.201386267513044, "kl": 0.09290932874682767, "context": 0.24784093180278433}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.07441307604312897, "loss_scale": 4.0, "retries": 0, "seconds": 83.8, "max_memory_gb": 17.17, "ce": 0.004965965577866882, "kl": 0.005115192044286232, "context": 0.007523561340349261}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.5466358065605164, "loss_scale": 4.0, "retries": 0, "seconds": 85.2, "max_memory_gb": 17.17, "ce": 0.018490998176275752, "kl": 0.012853852418629685, "context": 0.024917924820329063}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 2.7831788063049316, "loss_scale": 4.0, "retries": 0, "seconds": 86.4, "max_memory_gb": 17.17, "ce": 0.5928700248186942, "kl": 0.014130555166047998, "context": 0.5999353159568273}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 1.444258689880371, "loss_scale": 4.0, "retries": 0, "seconds": 87.7, "max_memory_gb": 17.17, "ce": 0.021218468973529525, "kl": 0.008572025035391562, "context": 0.025504482444375753}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.1603858917951584, "loss_scale": 4.0, "retries": 0, "seconds": 89.0, "max_memory_gb": 17.17, "ce": 0.006617259292397648, "kl": 0.006099528276536148, "context": 0.009667023448855616}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 0.9170620441436768, "loss_scale": 4.0, "retries": 0, "seconds": 90.6, "max_memory_gb": 17.17, "ce": 0.012051302765030414, "kl": 0.45768356474582106, "context": 0.24089308478869498}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 1.5058255195617676, "loss_scale": 4.0, "retries": 0, "seconds": 92.3, "max_memory_gb": 17.17, "ce": 0.1509953306231182, "kl": 0.03256799546215916, "context": 0.16727932791400235}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 0.9306952357292175, "loss_scale": 4.0, "retries": 0, "seconds": 93.5, "max_memory_gb": 17.17, "ce": 0.09707890982099343, "kl": 0.08924907631899259, "context": 0.14170344429658144}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.8275139331817627, "loss_scale": 4.0, "retries": 0, "seconds": 94.7, "max_memory_gb": 17.17, "ce": 0.5086710226751165, "kl": 0.8058794485405087, "context": 0.9116107488516718}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 2.3579518795013428, "loss_scale": 4.0, "retries": 0, "seconds": 96.1, "max_memory_gb": 17.17, "ce": 0.012871860039012972, "kl": 1.3251155448378995, "context": 0.6754296391736716}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.2582678496837616, "loss_scale": 4.0, "retries": 0, "seconds": 97.2, "max_memory_gb": 17.17, "ce": 0.012861082737799734, "kl": 0.011491603378090076, "context": 0.0186068844050169}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.6119132041931152, "loss_scale": 4.0, "retries": 0, "seconds": 98.4, "max_memory_gb": 17.17, "ce": 0.002176873487769626, "kl": 0.10656616369305993, "context": 0.0554599538736511}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.44089555740356445, "loss_scale": 4.0, "retries": 0, "seconds": 100.2, "max_memory_gb": 17.17, "ce": 0.014269659950514324, "kl": 0.01862380426200616, "context": 0.023581562069011852}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.43742385506629944, "loss_scale": 4.0, "retries": 0, "seconds": 101.5, "max_memory_gb": 17.17, "ce": 0.0025213187327608466, "kl": 0.05484399798024242, "context": 0.029943317254947033}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 0.8740825653076172, "loss_scale": 4.0, "retries": 0, "seconds": 102.9, "max_memory_gb": 17.17, "ce": 0.0846063612261787, "kl": 0.03859626912162639, "context": 0.10390449437545612}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.11758429557085037, "loss_scale": 4.0, "retries": 0, "seconds": 104.1, "max_memory_gb": 17.17, "ce": 0.009883007849566638, "kl": 0.0018818373100657482, "context": 0.010823926597367972}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.9584020376205444, "loss_scale": 4.0, "retries": 0, "seconds": 105.2, "max_memory_gb": 17.17, "ce": 0.11941685318015516, "kl": 0.5229495106614195, "context": 0.3808916188136209}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 1.62321937084198, "loss_scale": 4.0, "retries": 0, "seconds": 106.6, "max_memory_gb": 17.17, "ce": 0.11782979736744892, "kl": 0.03870111583364633, "context": 0.1371803537258529}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.25211235880851746, "loss_scale": 4.0, "retries": 0, "seconds": 107.8, "max_memory_gb": 17.17, "ce": 0.006151922072604066, "kl": 0.0022188610231737016, "context": 0.007261352817295119}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 1.1265932321548462, "loss_scale": 4.0, "retries": 0, "seconds": 109.0, "max_memory_gb": 17.17, "ce": 0.04188738271477632, "kl": 0.14593796338886023, "context": 0.11485636787256226}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 1.6373518705368042, "loss_scale": 4.0, "retries": 0, "seconds": 110.3, "max_memory_gb": 17.17, "ce": 0.008346136135514826, "kl": 0.5748422879260033, "context": 0.29576727864332497}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.2497423142194748, "loss_scale": 4.0, "retries": 0, "seconds": 111.4, "max_memory_gb": 17.17, "ce": 0.014747882712981664, "kl": 0.0027818478404810776, "context": 0.016138806982780807}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.4005718231201172, "loss_scale": 4.0, "retries": 0, "seconds": 112.6, "max_memory_gb": 17.17, "ce": 0.02585948453997844, "kl": 0.019943826511735097, "context": 0.03583139583861339}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.12856589257717133, "loss_scale": 4.0, "retries": 0, "seconds": 114.0, "max_memory_gb": 17.17, "ce": 0.012446634616935626, "kl": 0.0031313865329138935, "context": 0.014012327999807894}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 3.5623347759246826, "loss_scale": 4.0, "retries": 0, "seconds": 115.2, "max_memory_gb": 17.17, "ce": 0.48468451884400565, "kl": 0.24495427017609472, "context": 0.6071616436820477}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 3.842071056365967, "loss_scale": 4.0, "retries": 0, "seconds": 116.3, "max_memory_gb": 17.17, "ce": 0.10721504244429525, "kl": 0.21077507819791208, "context": 0.21260258120310027}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 2.6020750999450684, "loss_scale": 4.0, "retries": 0, "seconds": 117.6, "max_memory_gb": 17.17, "ce": 0.056773493452055845, "kl": 0.2623428281331144, "context": 0.18794491235166788}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 1.2988457679748535, "loss_scale": 4.0, "retries": 0, "seconds": 118.8, "max_memory_gb": 17.17, "ce": 0.0474896096275188, "kl": 0.018875233305152506, "context": 0.056927228084532544}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 3.280421018600464, "loss_scale": 4.0, "retries": 0, "seconds": 120.1, "max_memory_gb": 17.17, "ce": 0.46711250573571306, "kl": 0.01767841971013695, "context": 0.4759517257916741}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 2.397162914276123, "loss_scale": 4.0, "retries": 0, "seconds": 121.3, "max_memory_gb": 17.17, "ce": 0.3113943812495563, "kl": 0.006291564475759515, "context": 0.31454016352654435}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.6821309924125671, "loss_scale": 4.0, "retries": 0, "seconds": 122.5, "max_memory_gb": 17.17, "ce": 0.014076107763685286, "kl": 0.020374006766360253, "context": 0.0242631109431386}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.5081955194473267, "loss_scale": 4.0, "retries": 0, "seconds": 123.8, "max_memory_gb": 17.17, "ce": 0.0025209788364009, "kl": 0.09873975039590732, "context": 0.05189085577148944}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.16708023846149445, "loss_scale": 4.0, "retries": 0, "seconds": 125.0, "max_memory_gb": 17.17, "ce": 0.01640176075306954, "kl": 0.004167983801380615, "context": 0.018485753345885314}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 0.755021870136261, "loss_scale": 4.0, "retries": 0, "seconds": 126.2, "max_memory_gb": 17.17, "ce": 0.035114178084768355, "kl": 0.30748211701211403, "context": 0.1888552331365645}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.26930734515190125, "loss_scale": 4.0, "retries": 0, "seconds": 127.5, "max_memory_gb": 17.17, "ce": 0.0048207404324784875, "kl": 0.0328952991112601, "context": 0.021268390235491097}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.373121052980423, "loss_scale": 4.0, "retries": 0, "seconds": 128.6, "max_memory_gb": 17.17, "ce": 0.03199640254024416, "kl": 0.008878762018866837, "context": 0.03643578407354653}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.1916658580303192, "loss_scale": 4.0, "retries": 0, "seconds": 129.7, "max_memory_gb": 17.17, "ce": 0.011595440766541287, "kl": 0.006187464714457747, "context": 0.014689172894577496}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.28261756896972656, "loss_scale": 4.0, "retries": 0, "seconds": 130.7, "max_memory_gb": 17.17, "ce": 0.012303504045121372, "kl": 0.003975141022237949, "context": 0.014291074097855017}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 0.9790146946907043, "loss_scale": 4.0, "retries": 0, "seconds": 131.7, "max_memory_gb": 17.17, "ce": 0.022365347133018076, "kl": 0.335507938725641, "context": 0.19011931569548324}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.040450382977724075, "loss_scale": 4.0, "retries": 0, "seconds": 132.9, "max_memory_gb": 17.17, "ce": 0.0031036767322802916, "kl": 0.0005480218460434116, "context": 0.0033776875789044425}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.5746622085571289, "loss_scale": 4.0, "retries": 0, "seconds": 133.9, "max_memory_gb": 17.17, "ce": 0.021367163615650497, "kl": 0.17641379426027015, "context": 0.10957406446686946}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 2.130819320678711, "loss_scale": 4.0, "retries": 0, "seconds": 135.1, "max_memory_gb": 17.17, "ce": 0.37977294229494873, "kl": 0.00605055996743431, "context": 0.38279823132324964}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 3.281836986541748, "loss_scale": 4.0, "retries": 0, "seconds": 136.4, "max_memory_gb": 17.17, "ce": 0.2306905537698185, "kl": 0.13839797442778945, "context": 0.2998895344790071}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.5874468088150024, "loss_scale": 4.0, "retries": 0, "seconds": 137.6, "max_memory_gb": 17.17, "ce": 0.04086570558138192, "kl": 0.0027321946108713746, "context": 0.04223180375993252}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.021432388573884964, "loss_scale": 4.0, "retries": 0, "seconds": 138.8, "max_memory_gb": 17.17, "ce": 0.001551026536617428, "kl": 0.0007667687632419984, "context": 0.0019344108877703547}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.27118486166000366, "loss_scale": 4.0, "retries": 0, "seconds": 140.1, "max_memory_gb": 17.17, "ce": 0.0176551109761931, "kl": 0.0036845006579824258, "context": 0.01949736085953191}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 0.7141852378845215, "loss_scale": 4.0, "retries": 0, "seconds": 141.3, "max_memory_gb": 17.17, "ce": 0.011538912367541343, "kl": 0.08023086996399797, "context": 0.05165434768423438}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 2.4986696243286133, "loss_scale": 4.0, "retries": 0, "seconds": 142.6, "max_memory_gb": 17.17, "ce": 0.01561546770244604, "kl": 0.5090498586032481, "context": 0.27014039557252545}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 1.4029446840286255, "loss_scale": 4.0, "retries": 0, "seconds": 143.9, "max_memory_gb": 17.17, "ce": 0.04464544905931689, "kl": 0.07749074889579788, "context": 0.08339082333259284}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 1.1512526273727417, "loss_scale": 4.0, "retries": 0, "seconds": 145.2, "max_memory_gb": 17.17, "ce": 0.1070670868648449, "kl": 0.2897702107125042, "context": 0.2519521997164702}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.09886153042316437, "loss_scale": 4.0, "retries": 0, "seconds": 146.5, "max_memory_gb": 17.17, "ce": 0.00596352614229545, "kl": 0.003034628025488928, "context": 0.007480840082280338}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.2412543296813965, "loss_scale": 4.0, "retries": 0, "seconds": 147.8, "max_memory_gb": 17.17, "ce": 0.24635437988035847, "kl": 0.0038216801249291166, "context": 0.2482652174658142}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.18508268892765045, "loss_scale": 4.0, "retries": 0, "seconds": 148.8, "max_memory_gb": 17.17, "ce": 0.006858996712253429, "kl": 0.005900748426711289, "context": 0.00980937117128633}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 3.054328203201294, "loss_scale": 4.0, "retries": 0, "seconds": 150.0, "max_memory_gb": 17.17, "ce": 0.20927306835073978, "kl": 0.18429196950455662, "context": 0.3014190545072779}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.754391074180603, "loss_scale": 4.0, "retries": 0, "seconds": 151.0, "max_memory_gb": 17.17, "ce": 0.060990970421698876, "kl": 0.037643283048964804, "context": 0.07981260916858446}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.3162800073623657, "loss_scale": 4.0, "retries": 0, "seconds": 152.0, "max_memory_gb": 17.17, "ce": 0.14282962019206025, "kl": 0.02414022694756568, "context": 0.15489973017247394}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.8557931184768677, "loss_scale": 4.0, "retries": 0, "seconds": 153.2, "max_memory_gb": 17.17, "ce": 0.05415346333757043, "kl": 0.03052709816256538, "context": 0.06941701262257993}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 2.694399356842041, "loss_scale": 4.0, "retries": 0, "seconds": 154.2, "max_memory_gb": 17.17, "ce": 0.6206672183034243, "kl": 0.43998446940258873, "context": 0.8406594371772371}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.8046367764472961, "loss_scale": 4.0, "retries": 0, "seconds": 155.3, "max_memory_gb": 17.17, "ce": 0.1402921873377636, "kl": 0.06748987541141105, "context": 0.1740371212363243}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 2.864891767501831, "loss_scale": 4.0, "retries": 0, "seconds": 156.6, "max_memory_gb": 17.17, "ce": 0.6159880245104432, "kl": 0.09892389364540577, "context": 0.665449969470501}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.08815089613199234, "loss_scale": 4.0, "retries": 0, "seconds": 157.8, "max_memory_gb": 17.17, "ce": 0.005487053858814761, "kl": 0.00039372717947117053, "context": 0.00568391737760976}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 2.827547550201416, "loss_scale": 4.0, "retries": 0, "seconds": 159.1, "max_memory_gb": 17.17, "ce": 0.1238751479977509, "kl": 0.5261806278722361, "context": 0.38696545403217897}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 1.0425503253936768, "loss_scale": 4.0, "retries": 0, "seconds": 160.3, "max_memory_gb": 17.17, "ce": 0.09591161416028626, "kl": 0.03210037769531482, "context": 0.11196180112892762}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.608304500579834, "loss_scale": 4.0, "retries": 0, "seconds": 161.6, "max_memory_gb": 17.17, "ce": 0.03378935647197068, "kl": 0.2703564643088612, "context": 0.16896759264636785}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 3.8861067295074463, "loss_scale": 4.0, "retries": 0, "seconds": 163.0, "max_memory_gb": 17.17, "ce": 0.33713892847299576, "kl": 0.3580773160647368, "context": 0.5161775826709345}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 1.1512011289596558, "loss_scale": 4.0, "retries": 0, "seconds": 164.2, "max_memory_gb": 17.17, "ce": 0.15985321343760006, "kl": 0.034768034051012364, "context": 0.1772372304694727}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 2.40267276763916, "loss_scale": 4.0, "retries": 0, "seconds": 165.4, "max_memory_gb": 17.17, "ce": 0.3071957761676458, "kl": 0.021188274025917053, "context": 0.31778992142062634}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 0.9609863758087158, "loss_scale": 4.0, "retries": 0, "seconds": 166.8, "max_memory_gb": 17.17, "ce": 0.18374142680841032, "kl": 0.0037076562142601688, "context": 0.1855952519690618}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 0.4556732475757599, "loss_scale": 4.0, "retries": 0, "seconds": 168.0, "max_memory_gb": 17.17, "ce": 0.01698086003307253, "kl": 0.04720625471236417, "context": 0.04058398649794981}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 5.294233322143555, "loss_scale": 4.0, "retries": 0, "seconds": 169.4, "max_memory_gb": 17.17, "ce": 0.05476104351691902, "kl": 0.13201507719168148, "context": 0.12076858174987137}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 0.32915300130844116, "loss_scale": 4.0, "retries": 0, "seconds": 170.7, "max_memory_gb": 17.17, "ce": 0.015522540255915374, "kl": 0.015972096247423906, "context": 0.023508588375989348}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 2.929426670074463, "loss_scale": 4.0, "retries": 0, "seconds": 171.9, "max_memory_gb": 17.17, "ce": 0.10699718818068504, "kl": 0.15385087858885527, "context": 0.18392263120040298}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 1.1920210123062134, "loss_scale": 4.0, "retries": 0, "seconds": 173.3, "max_memory_gb": 17.17, "ce": 0.009910379783832468, "kl": 0.13341193451196887, "context": 0.07661634989199229}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.07776147872209549, "loss_scale": 4.0, "retries": 0, "seconds": 174.5, "max_memory_gb": 17.17, "ce": 0.005743500238168053, "kl": 0.0022461441294581164, "context": 0.006866572279250249}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.09034396708011627, "loss_scale": 4.0, "retries": 0, "seconds": 175.7, "max_memory_gb": 17.17, "ce": 0.00930380431964295, "kl": 0.0009014492097776383, "context": 0.009754529331985395}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 1.1075527667999268, "loss_scale": 4.0, "retries": 0, "seconds": 177.0, "max_memory_gb": 17.17, "ce": 0.06603611513855867, "kl": 0.021432439680211246, "context": 0.07675233529880643}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 3.195572853088379, "loss_scale": 4.0, "retries": 0, "seconds": 178.3, "max_memory_gb": 17.17, "ce": 0.32522376551060006, "kl": 0.027752866684750188, "context": 0.33910019358154386}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.8123297691345215, "loss_scale": 4.0, "retries": 0, "seconds": 179.4, "max_memory_gb": 17.17, "ce": 0.02788477309513837, "kl": 0.11799785622861236, "context": 0.0868837006855756}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.02985585480928421, "loss_scale": 4.0, "retries": 0, "seconds": 180.8, "max_memory_gb": 17.17, "ce": 0.001797216129489243, "kl": 0.002413736689959478, "context": 0.003004084515850991}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.19521063566207886, "loss_scale": 4.0, "retries": 0, "seconds": 182.0, "max_memory_gb": 17.17, "ce": 0.016917795277549885, "kl": 0.014794573027756996, "context": 0.024315081478562206}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.6739267110824585, "loss_scale": 4.0, "retries": 0, "seconds": 183.3, "max_memory_gb": 17.17, "ce": 0.37679406358802225, "kl": 0.4018937366345199, "context": 0.5777409137808718}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.1789889633655548, "loss_scale": 4.0, "retries": 0, "seconds": 184.5, "max_memory_gb": 17.17, "ce": 0.011502233915962279, "kl": 0.00834768005006481, "context": 0.015676074312068522}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 0.15881702303886414, "loss_scale": 4.0, "retries": 0, "seconds": 185.7, "max_memory_gb": 17.17, "ce": 0.015408583509270102, "kl": 0.002165800198326906, "context": 0.01649148331489414}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.47699716687202454, "loss_scale": 4.0, "retries": 0, "seconds": 187.0, "max_memory_gb": 17.17, "ce": 0.06305136188893812, "kl": 0.027444698781437182, "context": 0.07677371122554177}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.8349368572235107, "loss_scale": 4.0, "retries": 0, "seconds": 188.2, "max_memory_gb": 17.17, "ce": 0.03079710085876286, "kl": 0.03712945175357163, "context": 0.049361828569089994}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 1.2605782747268677, "loss_scale": 4.0, "retries": 0, "seconds": 189.4, "max_memory_gb": 17.17, "ce": 0.0030913799128029495, "kl": 0.1818804646281933, "context": 0.0940316112828441}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.7116131782531738, "loss_scale": 4.0, "retries": 0, "seconds": 190.7, "max_memory_gb": 17.17, "ce": 0.005767631344497204, "kl": 0.41413473370812426, "context": 0.21283499203855172}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.6025294661521912, "loss_scale": 4.0, "retries": 0, "seconds": 191.9, "max_memory_gb": 17.17, "ce": 0.053177632100414485, "kl": 0.08658116882725153, "context": 0.09646821650676429}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 1.6517914533615112, "loss_scale": 4.0, "retries": 0, "seconds": 193.1, "max_memory_gb": 17.17, "ce": 0.21944144669396337, "kl": 0.00943786275183811, "context": 0.2241603724542074}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 0.6943036913871765, "loss_scale": 4.0, "retries": 0, "seconds": 194.4, "max_memory_gb": 17.17, "ce": 0.016546467988518998, "kl": 0.010182627229823993, "context": 0.021637780708260834}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.3087701201438904, "loss_scale": 4.0, "retries": 0, "seconds": 195.6, "max_memory_gb": 17.17, "ce": 0.022008674684911966, "kl": 0.03483994327689288, "context": 0.03942864667624235}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.2585527896881104, "loss_scale": 4.0, "retries": 0, "seconds": 196.8, "max_memory_gb": 17.17, "ce": 0.1857576198526658, "kl": 0.10182060643273871, "context": 0.23666793038137257}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.5995170474052429, "loss_scale": 4.0, "retries": 0, "seconds": 198.1, "max_memory_gb": 17.17, "ce": 0.00627612293465063, "kl": 0.04495365312322974, "context": 0.028752949205227196}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 0.9323909282684326, "loss_scale": 4.0, "retries": 0, "seconds": 199.3, "max_memory_gb": 17.17, "ce": 0.07480444310931489, "kl": 0.21726589234458515, "context": 0.1834373808815144}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 1.7107361555099487, "loss_scale": 4.0, "retries": 0, "seconds": 200.6, "max_memory_gb": 17.17, "ce": 0.1674849969567731, "kl": 0.1180626374507483, "context": 0.22651631382177584}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 1.8067116737365723, "loss_scale": 4.0, "retries": 0, "seconds": 201.8, "max_memory_gb": 17.17, "ce": 0.0797150072758086, "kl": 0.11004758668423165, "context": 0.1347388006397523}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 5.6332268714904785, "loss_scale": 4.0, "retries": 0, "seconds": 202.9, "max_memory_gb": 17.17, "ce": 0.29321202464052476, "kl": 0.6876556018687552, "context": 0.6370398389117327}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 5.210349082946777, "loss_scale": 4.0, "retries": 0, "seconds": 204.3, "max_memory_gb": 17.17, "ce": 0.09696225282095838, "kl": 0.06565871858401806, "context": 0.12979161560360808}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.12937211990356445, "loss_scale": 4.0, "retries": 0, "seconds": 205.6, "max_memory_gb": 17.17, "ce": 0.010376231279224157, "kl": 0.006859211018308997, "context": 0.013805836555548012}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 2.2727277278900146, "loss_scale": 4.0, "retries": 0, "seconds": 206.8, "max_memory_gb": 17.17, "ce": 0.3545185399707407, "kl": 0.05162651113641914, "context": 0.3803317971760407}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.46135836839675903, "loss_scale": 4.0, "retries": 0, "seconds": 208.2, "max_memory_gb": 17.17, "ce": 0.030382653552806005, "kl": 0.00832592927326914, "context": 0.03454561805119738}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.13796089589595795, "loss_scale": 4.0, "retries": 0, "seconds": 209.3, "max_memory_gb": 17.17, "ce": 0.01110384201456327, "kl": 0.0016474379699502606, "context": 0.011927560743060894}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 1.5602887868881226, "loss_scale": 4.0, "retries": 0, "seconds": 210.7, "max_memory_gb": 17.17, "ce": 0.0749001419171691, "kl": 0.8899277336895466, "context": 0.5198640027083457}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.014445096254348755, "loss_scale": 4.0, "retries": 0, "seconds": 211.9, "max_memory_gb": 17.17, "ce": 0.0010106378758791834, "kl": 9.329621479992056e-05, "context": 0.0010572859755484387}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.6435036659240723, "loss_scale": 4.0, "retries": 0, "seconds": 213.1, "max_memory_gb": 17.17, "ce": 0.018612895633850712, "kl": 0.05048252030974254, "context": 0.043854154617292807}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.16465722024440765, "loss_scale": 4.0, "retries": 0, "seconds": 214.5, "max_memory_gb": 17.17, "ce": 0.008874238410498947, "kl": 0.01366595875879284, "context": 0.015707217738963664}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.25427448749542236, "loss_scale": 4.0, "retries": 0, "seconds": 215.7, "max_memory_gb": 17.17, "ce": 0.014464579202467576, "kl": 0.004460620039026253, "context": 0.016694889171049}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.4196285605430603, "loss_scale": 4.0, "retries": 0, "seconds": 216.9, "max_memory_gb": 17.17, "ce": 0.030276403631432913, "kl": 0.006881611846210944, "context": 0.03371720950235613}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 1.6956018209457397, "loss_scale": 4.0, "retries": 0, "seconds": 218.2, "max_memory_gb": 17.17, "ce": 0.09481946332380176, "kl": 0.03998259821673855, "context": 0.11481075966730714}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.07993241399526596, "loss_scale": 4.0, "retries": 0, "seconds": 219.4, "max_memory_gb": 17.17, "ce": 0.005659223403199576, "kl": 0.0006882960160510265, "context": 0.006003371570841409}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.9061622023582458, "loss_scale": 4.0, "retries": 0, "seconds": 220.6, "max_memory_gb": 17.17, "ce": 0.05332075052137952, "kl": 0.5940970680494502, "context": 0.35036929573107045}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 1.5465189218521118, "loss_scale": 4.0, "retries": 0, "seconds": 221.9, "max_memory_gb": 17.17, "ce": 0.1257781033637002, "kl": 0.017512889389763586, "context": 0.13453455176204443}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.1684731245040894, "loss_scale": 4.0, "retries": 0, "seconds": 223.1, "max_memory_gb": 17.17, "ce": 0.35476286173798144, "kl": 0.010165424801016343, "context": 0.3598455755854957}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.5981032252311707, "loss_scale": 4.0, "retries": 0, "seconds": 224.3, "max_memory_gb": 17.17, "ce": 0.042719373566797, "kl": 0.028304566458245972, "context": 0.056871658671298064}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 0.6254391074180603, "loss_scale": 4.0, "retries": 0, "seconds": 225.5, "max_memory_gb": 17.17, "ce": 0.0034774263913277537, "kl": 0.1067261725111166, "context": 0.05684051100979559}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.8958956003189087, "loss_scale": 4.0, "retries": 0, "seconds": 226.7, "max_memory_gb": 17.17, "ce": 0.06234698271146044, "kl": 0.23285732932345127, "context": 0.17877564317313954}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.040738288313150406, "loss_scale": 4.0, "retries": 0, "seconds": 228.0, "max_memory_gb": 17.17, "ce": 0.0028618536598514766, "kl": 0.00019844471171381883, "context": 0.0029610760539071634}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.06823340058326721, "loss_scale": 4.0, "retries": 0, "seconds": 229.2, "max_memory_gb": 17.17, "ce": 0.002553703132434748, "kl": 0.003091620950726792, "context": 0.004099513520486653}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.5814710855484009, "loss_scale": 4.0, "retries": 0, "seconds": 230.4, "max_memory_gb": 17.17, "ce": 0.0065237827875535, "kl": 0.10638205170994297, "context": 0.05971480713924393}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 2.522325038909912, "loss_scale": 4.0, "retries": 0, "seconds": 231.7, "max_memory_gb": 17.17, "ce": 0.06683402515773196, "kl": 0.10858193380045122, "context": 0.12112499558134004}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 0.9486365914344788, "loss_scale": 4.0, "retries": 0, "seconds": 232.9, "max_memory_gb": 17.17, "ce": 0.0175510984408902, "kl": 0.5463264142454136, "context": 0.29071430864860304}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.295774221420288, "loss_scale": 4.0, "retries": 0, "seconds": 234.1, "max_memory_gb": 17.17, "ce": 0.20112447087012697, "kl": 0.008123860228351987, "context": 0.20518640731461346}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 2.641160011291504, "loss_scale": 4.0, "retries": 0, "seconds": 235.4, "max_memory_gb": 17.17, "ce": 0.3041641759045888, "kl": 0.23300991077849176, "context": 0.4206691201252397}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.04864111915230751, "loss_scale": 4.0, "retries": 0, "seconds": 236.6, "max_memory_gb": 17.17, "ce": 0.0018163559579988942, "kl": 0.0028114517190260813, "context": 0.003222081810235977}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 5.933559894561768, "loss_scale": 4.0, "retries": 0, "seconds": 237.8, "max_memory_gb": 17.17, "ce": 0.23047295889409725, "kl": 0.15580966881861968, "context": 0.30837779329158366}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.04632568359375, "loss_scale": 4.0, "retries": 0, "seconds": 239.1, "max_memory_gb": 17.17, "ce": 0.38607733239769004, "kl": 0.029570783320195915, "context": 0.4008627305738628}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.049322132021188736, "loss_scale": 4.0, "retries": 0, "seconds": 240.3, "max_memory_gb": 17.17, "ce": 0.003050740764592774, "kl": 0.001213794427258108, "context": 0.0036576379934558645}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.8243358731269836, "loss_scale": 4.0, "retries": 0, "seconds": 241.6, "max_memory_gb": 17.17, "ce": 0.07677823869744316, "kl": 0.021341018060411443, "context": 0.08744874742114916}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.146212637424469, "loss_scale": 4.0, "retries": 0, "seconds": 242.7, "max_memory_gb": 17.17, "ce": 0.015151224884903058, "kl": 0.0009482129462412558, "context": 0.01562533140531741}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 1.167838454246521, "loss_scale": 4.0, "retries": 0, "seconds": 243.7, "max_memory_gb": 17.17, "ce": 0.003450788040936459, "kl": 0.20301477022553627, "context": 0.10495817214541603}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 3.124938726425171, "loss_scale": 4.0, "retries": 0, "seconds": 244.9, "max_memory_gb": 17.17, "ce": 0.05380453100951854, "kl": 0.48838376102958136, "context": 0.297996408000472}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.44780176877975464, "loss_scale": 4.0, "retries": 0, "seconds": 245.9, "max_memory_gb": 17.17, "ce": 0.08024595283495728, "kl": 0.007751952616672497, "context": 0.08412192935065832}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.1303153038024902, "loss_scale": 4.0, "retries": 0, "seconds": 247.1, "max_memory_gb": 17.17, "ce": 0.09129076582030393, "kl": 0.007893008447354077, "context": 0.09523726947372779}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 0.7195475101470947, "loss_scale": 4.0, "retries": 0, "seconds": 248.4, "max_memory_gb": 17.17, "ce": 0.22950143099296838, "kl": 0.027354706973710563, "context": 0.2431787842651829}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 1.2978359460830688, "loss_scale": 4.0, "retries": 0, "seconds": 249.6, "max_memory_gb": 17.17, "ce": 0.07226254639681429, "kl": 0.09290057927864837, "context": 0.11871283254004084}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.062246520072221756, "loss_scale": 4.0, "retries": 0, "seconds": 250.8, "max_memory_gb": 17.17, "ce": 0.003523500156006776, "kl": 0.0028062304845661856, "context": 0.0049266154383076355}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.44198665022850037, "loss_scale": 4.0, "retries": 0, "seconds": 252.0, "max_memory_gb": 17.17, "ce": 0.021700417586544063, "kl": 0.02096077705209609, "context": 0.03218080547230784}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.2757048010826111, "loss_scale": 4.0, "retries": 0, "seconds": 253.2, "max_memory_gb": 17.17, "ce": 0.021502008312381804, "kl": 0.0025768933701328933, "context": 0.02279045491013676}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 1.2451481819152832, "loss_scale": 4.0, "retries": 0, "seconds": 254.5, "max_memory_gb": 17.17, "ce": 0.007156004910939373, "kl": 0.38828034654034127, "context": 0.20129618109785952}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.12823623418807983, "loss_scale": 4.0, "retries": 0, "seconds": 255.7, "max_memory_gb": 17.17, "ce": 0.00658609185484238, "kl": 0.002333184153030743, "context": 0.007752684279694222}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.4518103003501892, "loss_scale": 4.0, "retries": 0, "seconds": 256.8, "max_memory_gb": 17.17, "ce": 0.08456151420250535, "kl": 0.04904958640690893, "context": 0.10908630909398198}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.056558698415756226, "loss_scale": 4.0, "retries": 0, "seconds": 258.1, "max_memory_gb": 17.17, "ce": 0.003663602750748396, "kl": 0.0010281442973791854, "context": 0.004177674985839985}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 1.7619740962982178, "loss_scale": 4.0, "retries": 0, "seconds": 259.3, "max_memory_gb": 17.17, "ce": 0.20097908948082477, "kl": 0.05240908049745485, "context": 0.22718362975865602}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 3.505138397216797, "loss_scale": 4.0, "retries": 0, "seconds": 260.5, "max_memory_gb": 17.17, "ce": 0.44262218108633533, "kl": 0.09613825957740119, "context": 0.49069129588315263}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.5610507726669312, "loss_scale": 4.0, "retries": 0, "seconds": 261.8, "max_memory_gb": 17.17, "ce": 0.12706695566885173, "kl": 0.058898196744848974, "context": 0.15651605834136717}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 3.047316551208496, "loss_scale": 4.0, "retries": 0, "seconds": 262.9, "max_memory_gb": 17.17, "ce": 0.12936269403144252, "kl": 0.9295813765238563, "context": 0.5941533898003399}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.7649290561676025, "loss_scale": 4.0, "retries": 0, "seconds": 264.3, "max_memory_gb": 17.17, "ce": 0.19916486492729746, "kl": 0.023119812247387017, "context": 0.21072476933477446}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.7665789127349854, "loss_scale": 4.0, "retries": 0, "seconds": 265.5, "max_memory_gb": 17.17, "ce": 0.2949084380234126, "kl": 0.06374094705097377, "context": 0.32677890904597007}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 2.1671431064605713, "loss_scale": 4.0, "retries": 0, "seconds": 266.6, "max_memory_gb": 17.17, "ce": 0.48375505444710143, "kl": 0.01914081639779397, "context": 0.49332545531797223}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.5690078139305115, "loss_scale": 4.0, "retries": 0, "seconds": 267.8, "max_memory_gb": 17.17, "ce": 0.02434516599169001, "kl": 0.07239638781175017, "context": 0.06054336065426469}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.05337677150964737, "loss_scale": 4.0, "retries": 0, "seconds": 268.9, "max_memory_gb": 17.18, "ce": 0.003819125660811551, "kl": 0.0017659068789726007, "context": 0.004702079182607122}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 1.178498387336731, "loss_scale": 4.0, "retries": 0, "seconds": 270.0, "max_memory_gb": 17.18, "ce": 0.17543486057547852, "kl": 0.18203949369490147, "context": 0.2664546128362417}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 2.95202898979187, "loss_scale": 4.0, "retries": 0, "seconds": 271.3, "max_memory_gb": 17.18, "ce": 0.012707378511549905, "kl": 0.22353706062858691, "context": 0.12447590939700603}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 0.21131902933120728, "loss_scale": 4.0, "retries": 0, "seconds": 272.5, "max_memory_gb": 17.18, "ce": 0.017579044826561585, "kl": 0.0013792574591207085, "context": 0.018268673768034205}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.632189154624939, "loss_scale": 4.0, "retries": 0, "seconds": 273.7, "max_memory_gb": 17.18, "ce": 0.004034558958665002, "kl": 0.07008964024862507, "context": 0.03907937815529294}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 1.0086243152618408, "loss_scale": 4.0, "retries": 0, "seconds": 275.0, "max_memory_gb": 17.18, "ce": 0.05522593486239202, "kl": 0.049809514472144656, "context": 0.08013069134904072}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 2.502854585647583, "loss_scale": 4.0, "retries": 0, "seconds": 276.2, "max_memory_gb": 17.18, "ce": 0.1565698280755896, "kl": 0.006936083365872037, "context": 0.1600378711882513}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 5.548633098602295, "loss_scale": 4.0, "retries": 0, "seconds": 277.3, "max_memory_gb": 17.18, "ce": 0.047085040947422385, "kl": 0.04856226566789701, "context": 0.07136617350624874}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.19157926738262177, "loss_scale": 4.0, "retries": 0, "seconds": 278.7, "max_memory_gb": 17.18, "ce": 0.006046785551006906, "kl": 0.017169886064948514, "context": 0.014631728525273502}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.4631129503250122, "loss_scale": 4.0, "retries": 0, "seconds": 279.9, "max_memory_gb": 17.18, "ce": 0.026005764797446318, "kl": 0.007231184168631444, "context": 0.029621356137795374}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.3758312463760376, "loss_scale": 4.0, "retries": 0, "seconds": 281.2, "max_memory_gb": 17.18, "ce": 0.017005287852953188, "kl": 0.009691232286058948, "context": 0.02185090350394603}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.34215086698532104, "loss_scale": 4.0, "retries": 0, "seconds": 282.4, "max_memory_gb": 17.18, "ce": 0.009326530183898285, "kl": 0.03239056846359745, "context": 0.025521814823150635}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 1.0826148986816406, "loss_scale": 4.0, "retries": 0, "seconds": 283.6, "max_memory_gb": 17.18, "ce": 0.07236800136161037, "kl": 0.03030429399177592, "context": 0.0875201483577257}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 2.051370143890381, "loss_scale": 4.0, "retries": 0, "seconds": 284.9, "max_memory_gb": 17.18, "ce": 0.44005225965520367, "kl": 0.25842736632330343, "context": 0.5692659547785297}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.7399893999099731, "loss_scale": 4.0, "retries": 0, "seconds": 286.1, "max_memory_gb": 17.18, "ce": 0.044640858977800235, "kl": 0.08555891769356094, "context": 0.08742031722795218}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.49993401765823364, "loss_scale": 4.0, "retries": 0, "seconds": 287.3, "max_memory_gb": 17.18, "ce": 0.02978266723221168, "kl": 0.006690653186524287, "context": 0.0331279928504955}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.698851466178894, "loss_scale": 4.0, "retries": 0, "seconds": 288.6, "max_memory_gb": 17.18, "ce": 0.02391612168867141, "kl": 0.0940285058459267, "context": 0.07093037501908839}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.3426136076450348, "loss_scale": 4.0, "retries": 0, "seconds": 289.8, "max_memory_gb": 17.18, "ce": 0.01745124172884971, "kl": 0.0071104494118117145, "context": 0.021006465423852205}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 1.7811102867126465, "loss_scale": 4.0, "retries": 0, "seconds": 291.1, "max_memory_gb": 17.18, "ce": 0.32454464345937595, "kl": 0.019715437044396822, "context": 0.33440237503964454}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.19908033311367035, "loss_scale": 4.0, "retries": 0, "seconds": 292.2, "max_memory_gb": 17.18, "ce": 0.01341940465499647, "kl": 0.017286397800489794, "context": 0.02206260379170999}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 2.3279130458831787, "loss_scale": 4.0, "retries": 0, "seconds": 293.4, "max_memory_gb": 17.18, "ce": 0.06469505114364438, "kl": 0.018591526258205704, "context": 0.07399081136099994}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.28617367148399353, "loss_scale": 4.0, "retries": 0, "seconds": 294.7, "max_memory_gb": 17.18, "ce": 0.0237735746341059, "kl": 0.004976131465809885, "context": 0.026261639344738796}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.45461148023605347, "loss_scale": 4.0, "retries": 0, "seconds": 295.9, "max_memory_gb": 17.18, "ce": 0.10625060275197029, "kl": 0.01303075532814546, "context": 0.11276597925461829}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 0.3891313076019287, "loss_scale": 4.0, "retries": 0, "seconds": 297.1, "max_memory_gb": 17.18, "ce": 0.02639886731049046, "kl": 0.03505999696426443, "context": 0.043928865692578256}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 1.695479393005371, "loss_scale": 4.0, "retries": 0, "seconds": 298.4, "max_memory_gb": 17.18, "ce": 0.10464435670292005, "kl": 0.03235976247378858, "context": 0.12082424061372876}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.2013567090034485, "loss_scale": 4.0, "retries": 0, "seconds": 299.5, "max_memory_gb": 17.18, "ce": 0.013023760591750033, "kl": 0.01403267186424273, "context": 0.02004009613301605}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.047276709228754044, "loss_scale": 4.0, "retries": 0, "seconds": 300.7, "max_memory_gb": 17.18, "ce": 0.0032212437654379755, "kl": 0.0010471214864082867, "context": 0.0037448044458869845}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 1.6559559106826782, "loss_scale": 4.0, "retries": 0, "seconds": 302.1, "max_memory_gb": 17.18, "ce": 0.04062908297782997, "kl": 0.21428132875644224, "context": 0.14776974455890013}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 1.5209842920303345, "loss_scale": 4.0, "retries": 0, "seconds": 303.3, "max_memory_gb": 17.18, "ce": 0.3325035207817564, "kl": 0.028033666872943286, "context": 0.3465203540981747}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.026902783662080765, "loss_scale": 4.0, "retries": 0, "seconds": 304.6, "max_memory_gb": 17.18, "ce": 0.001908557940623723, "kl": 0.0003727860748767853, "context": 0.0020949510362697765}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 1.2038196325302124, "loss_scale": 4.0, "retries": 0, "seconds": 305.8, "max_memory_gb": 17.18, "ce": 0.011052819507312961, "kl": 0.1878469399016467, "context": 0.10497628865414299}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.2829206585884094, "loss_scale": 4.0, "retries": 0, "seconds": 306.9, "max_memory_gb": 17.18, "ce": 0.02076596417464316, "kl": 0.00727799121523276, "context": 0.024404959636740386}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 1.3247394561767578, "loss_scale": 4.0, "retries": 0, "seconds": 308.3, "max_memory_gb": 17.18, "ce": 0.08182707289233804, "kl": 0.06912430201191455, "context": 0.11638922663405538}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.167165607213974, "loss_scale": 4.0, "retries": 0, "seconds": 309.5, "max_memory_gb": 17.18, "ce": 0.010535858105868101, "kl": 0.006916934027685784, "context": 0.013994325185194612}

runs/fixed_internvl/context/Random8_8102_s13/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Random8_8102_s13/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.10.mlp.down_proj": 144,
+    "model.language_model.layers.10.mlp.gate_proj": 144,
+    "model.language_model.layers.10.mlp.up_proj": 144,
+    "model.language_model.layers.10.self_attn.k_proj": 144,
+    "model.language_model.layers.10.self_attn.o_proj": 144,
+    "model.language_model.layers.10.self_attn.q_proj": 144,
+    "model.language_model.layers.10.self_attn.v_proj": 144,
+    "model.language_model.layers.11.mlp.down_proj": 144,
+    "model.language_model.layers.11.mlp.gate_proj": 144,
+    "model.language_model.layers.11.mlp.up_proj": 144,
+    "model.language_model.layers.11.self_attn.k_proj": 144,
+    "model.language_model.layers.11.self_attn.o_proj": 144,
+    "model.language_model.layers.11.self_attn.q_proj": 144,
+    "model.language_model.layers.11.self_attn.v_proj": 144,
+    "model.language_model.layers.15.mlp.down_proj": 144,
+    "model.language_model.layers.15.mlp.gate_proj": 144,
+    "model.language_model.layers.15.mlp.up_proj": 144,
+    "model.language_model.layers.15.self_attn.k_proj": 144,
+    "model.language_model.layers.15.self_attn.o_proj": 144,
+    "model.language_model.layers.15.self_attn.q_proj": 144,
+    "model.language_model.layers.15.self_attn.v_proj": 144,
+    "model.language_model.layers.18.mlp.down_proj": 144,
+    "model.language_model.layers.18.mlp.gate_proj": 144,
+    "model.language_model.layers.18.mlp.up_proj": 144,
+    "model.language_model.layers.18.self_attn.k_proj": 144,
+    "model.language_model.layers.18.self_attn.o_proj": 144,
+    "model.language_model.layers.18.self_attn.q_proj": 144,
+    "model.language_model.layers.18.self_attn.v_proj": 144,
+    "model.language_model.layers.32.mlp.down_proj": 144,
+    "model.language_model.layers.32.mlp.gate_proj": 144,
+    "model.language_model.layers.32.mlp.up_proj": 144,
+    "model.language_model.layers.32.self_attn.k_proj": 144,
+    "model.language_model.layers.32.self_attn.o_proj": 144,
+    "model.language_model.layers.32.self_attn.q_proj": 144,
+    "model.language_model.layers.32.self_attn.v_proj": 144,
+    "model.language_model.layers.34.mlp.down_proj": 144,
+    "model.language_model.layers.34.mlp.gate_proj": 144,
+    "model.language_model.layers.34.mlp.up_proj": 144,
+    "model.language_model.layers.34.self_attn.k_proj": 144,
+    "model.language_model.layers.34.self_attn.o_proj": 144,
+    "model.language_model.layers.34.self_attn.q_proj": 144,
+    "model.language_model.layers.34.self_attn.v_proj": 144,
+    "model.language_model.layers.4.mlp.down_proj": 144,
+    "model.language_model.layers.4.mlp.gate_proj": 144,
+    "model.language_model.layers.4.mlp.up_proj": 144,
+    "model.language_model.layers.4.self_attn.k_proj": 144,
+    "model.language_model.layers.4.self_attn.o_proj": 144,
+    "model.language_model.layers.4.self_attn.q_proj": 144,
+    "model.language_model.layers.4.self_attn.v_proj": 144,
+    "model.language_model.layers.9.mlp.down_proj": 144,
+    "model.language_model.layers.9.mlp.gate_proj": 144,
+    "model.language_model.layers.9.mlp.up_proj": 144,
+    "model.language_model.layers.9.self_attn.k_proj": 144,
+    "model.language_model.layers.9.self_attn.o_proj": 144,
+    "model.language_model.layers.9.self_attn.q_proj": 144,
+    "model.language_model.layers.9.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.10.mlp.down_proj": 72,
+    "model.language_model.layers.10.mlp.gate_proj": 72,
+    "model.language_model.layers.10.mlp.up_proj": 72,
+    "model.language_model.layers.10.self_attn.k_proj": 72,
+    "model.language_model.layers.10.self_attn.o_proj": 72,
+    "model.language_model.layers.10.self_attn.q_proj": 72,
+    "model.language_model.layers.10.self_attn.v_proj": 72,
+    "model.language_model.layers.11.mlp.down_proj": 72,
+    "model.language_model.layers.11.mlp.gate_proj": 72,
+    "model.language_model.layers.11.mlp.up_proj": 72,
+    "model.language_model.layers.11.self_attn.k_proj": 72,
+    "model.language_model.layers.11.self_attn.o_proj": 72,
+    "model.language_model.layers.11.self_attn.q_proj": 72,
+    "model.language_model.layers.11.self_attn.v_proj": 72,
+    "model.language_model.layers.15.mlp.down_proj": 72,
+    "model.language_model.layers.15.mlp.gate_proj": 72,
+    "model.language_model.layers.15.mlp.up_proj": 72,
+    "model.language_model.layers.15.self_attn.k_proj": 72,
+    "model.language_model.layers.15.self_attn.o_proj": 72,
+    "model.language_model.layers.15.self_attn.q_proj": 72,
+    "model.language_model.layers.15.self_attn.v_proj": 72,
+    "model.language_model.layers.18.mlp.down_proj": 72,
+    "model.language_model.layers.18.mlp.gate_proj": 72,
+    "model.language_model.layers.18.mlp.up_proj": 72,
+    "model.language_model.layers.18.self_attn.k_proj": 72,
+    "model.language_model.layers.18.self_attn.o_proj": 72,
+    "model.language_model.layers.18.self_attn.q_proj": 72,
+    "model.language_model.layers.18.self_attn.v_proj": 72,
+    "model.language_model.layers.32.mlp.down_proj": 72,
+    "model.language_model.layers.32.mlp.gate_proj": 72,
+    "model.language_model.layers.32.mlp.up_proj": 72,
+    "model.language_model.layers.32.self_attn.k_proj": 72,
+    "model.language_model.layers.32.self_attn.o_proj": 72,
+    "model.language_model.layers.32.self_attn.q_proj": 72,
+    "model.language_model.layers.32.self_attn.v_proj": 72,
+    "model.language_model.layers.34.mlp.down_proj": 72,
+    "model.language_model.layers.34.mlp.gate_proj": 72,
+    "model.language_model.layers.34.mlp.up_proj": 72,
+    "model.language_model.layers.34.self_attn.k_proj": 72,
+    "model.language_model.layers.34.self_attn.o_proj": 72,
+    "model.language_model.layers.34.self_attn.q_proj": 72,
+    "model.language_model.layers.34.self_attn.v_proj": 72,
+    "model.language_model.layers.4.mlp.down_proj": 72,
+    "model.language_model.layers.4.mlp.gate_proj": 72,
+    "model.language_model.layers.4.mlp.up_proj": 72,
+    "model.language_model.layers.4.self_attn.k_proj": 72,
+    "model.language_model.layers.4.self_attn.o_proj": 72,
+    "model.language_model.layers.4.self_attn.q_proj": 72,
+    "model.language_model.layers.4.self_attn.v_proj": 72,
+    "model.language_model.layers.9.mlp.down_proj": 72,
+    "model.language_model.layers.9.mlp.gate_proj": 72,
+    "model.language_model.layers.9.mlp.up_proj": 72,
+    "model.language_model.layers.9.self_attn.k_proj": 72,
+    "model.language_model.layers.9.self_attn.o_proj": 72,
+    "model.language_model.layers.9.self_attn.q_proj": 72,
+    "model.language_model.layers.9.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.11.mlp.down_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.4.mlp.down_proj",
+    "model.language_model.layers.9.self_attn.k_proj",
+    "model.language_model.layers.11.self_attn.q_proj",
+    "model.language_model.layers.11.self_attn.o_proj",
+    "model.language_model.layers.10.self_attn.v_proj",
+    "model.language_model.layers.34.mlp.down_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.34.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.32.mlp.down_proj",
+    "model.language_model.layers.4.self_attn.v_proj",
+    "model.language_model.layers.4.mlp.up_proj",
+    "model.language_model.layers.11.self_attn.k_proj",
+    "model.language_model.layers.10.self_attn.q_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.32.mlp.gate_proj",
+    "model.language_model.layers.9.self_attn.q_proj",
+    "model.language_model.layers.10.self_attn.k_proj",
+    "model.language_model.layers.4.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.4.mlp.gate_proj",
+    "model.language_model.layers.32.self_attn.v_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.10.mlp.down_proj",
+    "model.language_model.layers.32.self_attn.k_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.32.mlp.up_proj",
+    "model.language_model.layers.4.self_attn.k_proj",
+    "model.language_model.layers.4.self_attn.o_proj",
+    "model.language_model.layers.11.mlp.up_proj",
+    "model.language_model.layers.34.mlp.gate_proj",
+    "model.language_model.layers.9.mlp.up_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.34.self_attn.v_proj",
+    "model.language_model.layers.10.mlp.up_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.34.mlp.up_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.9.self_attn.v_proj",
+    "model.language_model.layers.32.self_attn.q_proj",
+    "model.language_model.layers.32.self_attn.o_proj",
+    "model.language_model.layers.10.self_attn.o_proj",
+    "model.language_model.layers.11.self_attn.v_proj",
+    "model.language_model.layers.34.self_attn.k_proj",
+    "model.language_model.layers.11.mlp.gate_proj",
+    "model.language_model.layers.9.mlp.down_proj",
+    "model.language_model.layers.9.mlp.gate_proj",
+    "model.language_model.layers.34.self_attn.o_proj",
+    "model.language_model.layers.10.mlp.gate_proj",
+    "model.language_model.layers.9.self_attn.o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Random8_8102_s13/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 13,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Random8_8102",
+    "layers": [
+      4,
+      9,
+      10,
+      11,
+      15,
+      18,
+      32,
+      34
+    ],
+    "ranks": {
+      "4": 72,
+      "9": 72,
+      "10": 72,
+      "11": 72,
+      "15": 72,
+      "18": 72,
+      "32": 72,
+      "34": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Random8_8102_s13/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 5.263116359710693, "loss_scale": 4.0, "retries": 0, "seconds": 2.1, "max_memory_gb": 16.82, "ce": 0.5227766726748087, "kl": 0.08654405153356493, "context": 0.5660487064160407}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 6.862936973571777, "loss_scale": 4.0, "retries": 0, "seconds": 3.5, "max_memory_gb": 17.16, "ce": 0.6328521318937419, "kl": 0.4680680775019255, "context": 0.8668861778642167}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 17.979354858398438, "loss_scale": 4.0, "retries": 0, "seconds": 4.7, "max_memory_gb": 17.16, "ce": 0.969939336180687, "kl": 0.06931963562965393, "context": 1.004599168896675}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 6.438618183135986, "loss_scale": 4.0, "retries": 0, "seconds": 6.1, "max_memory_gb": 17.16, "ce": 0.26388866312481696, "kl": 0.45484672344764476, "context": 0.49131202091666637}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 0.24095110595226288, "loss_scale": 4.0, "retries": 0, "seconds": 7.3, "max_memory_gb": 17.16, "ce": 0.009497352783000679, "kl": 0.008548446639906615, "context": 0.013771576159342658}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 3.126824378967285, "loss_scale": 4.0, "retries": 0, "seconds": 8.5, "max_memory_gb": 17.16, "ce": 0.15727824479108676, "kl": 0.08098078018520027, "context": 0.19776863255538046}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 2.5231008529663086, "loss_scale": 4.0, "retries": 0, "seconds": 9.8, "max_memory_gb": 17.16, "ce": 0.10084157861638232, "kl": 0.06539477239584812, "context": 0.13353896434819035}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 1.6147964000701904, "loss_scale": 4.0, "retries": 0, "seconds": 11.0, "max_memory_gb": 17.16, "ce": 0.04121518414467573, "kl": 0.21006035385653377, "context": 0.14624536223709583}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 9.813679695129395, "loss_scale": 4.0, "retries": 0, "seconds": 12.2, "max_memory_gb": 17.16, "ce": 0.09775365004315972, "kl": 0.17620722739957273, "context": 0.18585726292803884}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 0.6317757368087769, "loss_scale": 4.0, "retries": 0, "seconds": 13.5, "max_memory_gb": 17.16, "ce": 0.005100538495753426, "kl": 0.021415255032479763, "context": 0.015808166412170976}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.13230548799037933, "loss_scale": 4.0, "retries": 0, "seconds": 14.7, "max_memory_gb": 17.16, "ce": 0.0050988665243494324, "kl": 0.004051549487940065, "context": 0.007124641444534063}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 0.22534863650798798, "loss_scale": 4.0, "retries": 0, "seconds": 15.9, "max_memory_gb": 17.16, "ce": 0.00973196094719242, "kl": 0.0022398243827410624, "context": 0.010851872905732307}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.049520134925842285, "loss_scale": 4.0, "retries": 0, "seconds": 17.2, "max_memory_gb": 17.16, "ce": 0.0014299237300292589, "kl": 0.0004656926994357491, "context": 0.0016627701043034904}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 2.610543966293335, "loss_scale": 4.0, "retries": 0, "seconds": 18.3, "max_memory_gb": 17.16, "ce": 0.0610187416896224, "kl": 0.7430775426328182, "context": 0.43255751533433795}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 0.9481384754180908, "loss_scale": 4.0, "retries": 0, "seconds": 19.7, "max_memory_gb": 17.16, "ce": 0.008845725230571588, "kl": 0.34300274082779936, "context": 0.1803470984384603}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 0.03758431226015091, "loss_scale": 4.0, "retries": 0, "seconds": 20.8, "max_memory_gb": 17.16, "ce": 0.00192821540258592, "kl": 0.0006720462774865155, "context": 0.002264238559291698}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 2.1312921047210693, "loss_scale": 4.0, "retries": 0, "seconds": 22.0, "max_memory_gb": 17.16, "ce": 0.023286945288418792, "kl": 0.7673947975636111, "context": 0.4069843534962274}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 0.821559488773346, "loss_scale": 4.0, "retries": 0, "seconds": 23.3, "max_memory_gb": 17.16, "ce": 0.007097764035279397, "kl": 0.26258266078002634, "context": 0.13838909447804326}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.43646761775016785, "loss_scale": 4.0, "retries": 0, "seconds": 24.5, "max_memory_gb": 17.16, "ce": 0.003811026701441733, "kl": 0.08621774838138663, "context": 0.04691990034552873}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.11461333930492401, "loss_scale": 4.0, "retries": 0, "seconds": 25.8, "max_memory_gb": 17.16, "ce": 0.0009150718169621541, "kl": 0.008000150431143993, "context": 0.004915147135761799}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.052430376410484314, "loss_scale": 4.0, "retries": 0, "seconds": 27.1, "max_memory_gb": 17.16, "ce": 0.0006221598309821275, "kl": 0.0038133339567139046, "context": 0.002528826809793827}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 1.3927582502365112, "loss_scale": 4.0, "retries": 0, "seconds": 28.4, "max_memory_gb": 17.16, "ce": 0.06506125733722001, "kl": 0.049621069916611305, "context": 0.08987179066753015}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 2.5553290843963623, "loss_scale": 4.0, "retries": 0, "seconds": 29.6, "max_memory_gb": 17.16, "ce": 0.0003446518658165587, "kl": 0.43825503942915134, "context": 0.21947217338856717}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.5498982667922974, "loss_scale": 4.0, "retries": 0, "seconds": 31.0, "max_memory_gb": 17.16, "ce": 0.007909468455181923, "kl": 0.042147858253883896, "context": 0.028983397714910097}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 2.4307687282562256, "loss_scale": 4.0, "retries": 0, "seconds": 32.2, "max_memory_gb": 17.16, "ce": 0.1872299399219628, "kl": 0.04246456013061106, "context": 0.20846222643740475}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 1.0050235986709595, "loss_scale": 4.0, "retries": 0, "seconds": 33.6, "max_memory_gb": 17.16, "ce": 0.040711153797019506, "kl": 0.5398570944635139, "context": 0.31063970473769587}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 1.334039330482483, "loss_scale": 4.0, "retries": 0, "seconds": 34.7, "max_memory_gb": 17.16, "ce": 0.09525276761269197, "kl": 0.2739564026123844, "context": 0.2322309761948418}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 2.7819106578826904, "loss_scale": 4.0, "retries": 0, "seconds": 36.0, "max_memory_gb": 17.16, "ce": 0.1173777596814034, "kl": 0.3310404696967453, "context": 0.28289799951016903}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 4.391539096832275, "loss_scale": 4.0, "retries": 0, "seconds": 37.3, "max_memory_gb": 17.16, "ce": 0.4241742878803052, "kl": 0.15210161253344268, "context": 0.5002250820398331}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.11213552206754684, "loss_scale": 4.0, "retries": 0, "seconds": 38.5, "max_memory_gb": 17.16, "ce": 0.002257209474919364, "kl": 0.005468873743666336, "context": 0.004991646332200617}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 6.393899440765381, "loss_scale": 4.0, "retries": 0, "seconds": 39.8, "max_memory_gb": 17.16, "ce": 0.671685838024132, "kl": 0.12057360965991393, "context": 0.7319726390996948}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 0.46532729268074036, "loss_scale": 4.0, "retries": 0, "seconds": 41.1, "max_memory_gb": 17.16, "ce": 0.016925325617194176, "kl": 0.010084741246828344, "context": 0.02196769620059058}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 3.9362752437591553, "loss_scale": 4.0, "retries": 0, "seconds": 42.3, "max_memory_gb": 17.16, "ce": 0.28320609260117635, "kl": 0.08934405696345493, "context": 0.32787811604794115}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.5001449584960938, "loss_scale": 4.0, "retries": 0, "seconds": 43.5, "max_memory_gb": 17.16, "ce": 0.0070222761714830995, "kl": 0.7368388167378725, "context": 0.37544167379383}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.14007660746574402, "loss_scale": 4.0, "retries": 0, "seconds": 44.8, "max_memory_gb": 17.16, "ce": 0.004121186713746283, "kl": 0.00034351771591900615, "context": 0.004292945486668032}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 2.1026952266693115, "loss_scale": 4.0, "retries": 0, "seconds": 46.0, "max_memory_gb": 17.16, "ce": 0.21650632738601416, "kl": 0.04435659744194709, "context": 0.2386846251320094}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 1.975140929222107, "loss_scale": 4.0, "retries": 0, "seconds": 47.3, "max_memory_gb": 17.16, "ce": 0.1109603435616009, "kl": 0.39952093409374356, "context": 0.310720803681761}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 1.5865302085876465, "loss_scale": 4.0, "retries": 0, "seconds": 48.5, "max_memory_gb": 17.16, "ce": 0.09177054278552532, "kl": 0.37860812508733943, "context": 0.28107461030595005}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.2013107538223267, "loss_scale": 4.0, "retries": 0, "seconds": 49.7, "max_memory_gb": 17.16, "ce": 0.06260693550575525, "kl": 0.0065895124498638324, "context": 0.06590168896946125}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.6874438524246216, "loss_scale": 4.0, "retries": 0, "seconds": 51.0, "max_memory_gb": 17.16, "ce": 0.03647817962337285, "kl": 0.039394796069245785, "context": 0.05617557722143829}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 2.9889326095581055, "loss_scale": 4.0, "retries": 0, "seconds": 52.2, "max_memory_gb": 17.16, "ce": 0.5364518160931766, "kl": 0.09551938669756055, "context": 0.5842114984989166}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 0.1327149122953415, "loss_scale": 4.0, "retries": 0, "seconds": 53.4, "max_memory_gb": 17.16, "ce": 0.00913839036365971, "kl": 0.0006879791953906533, "context": 0.009482379857217893}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 0.07726874947547913, "loss_scale": 4.0, "retries": 0, "seconds": 54.7, "max_memory_gb": 17.16, "ce": 0.005775001482106745, "kl": 0.0008654679004393984, "context": 0.006207735335920006}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 12.375092506408691, "loss_scale": 4.0, "retries": 0, "seconds": 55.9, "max_memory_gb": 17.16, "ce": 0.1056951701611979, "kl": 0.6889594723979826, "context": 0.4501749212795403}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 3.6712212562561035, "loss_scale": 4.0, "retries": 0, "seconds": 57.1, "max_memory_gb": 17.16, "ce": 0.46345598611515015, "kl": 0.1345431849185843, "context": 0.5307275790255517}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 1.1075832843780518, "loss_scale": 4.0, "retries": 0, "seconds": 58.4, "max_memory_gb": 17.16, "ce": 0.06473431352060288, "kl": 0.11601057316875085, "context": 0.12273959966842085}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 0.8088240623474121, "loss_scale": 4.0, "retries": 0, "seconds": 59.6, "max_memory_gb": 17.16, "ce": 0.03006902626657393, "kl": 0.1275267186574638, "context": 0.09383238374721259}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.06482122838497162, "loss_scale": 4.0, "retries": 0, "seconds": 60.9, "max_memory_gb": 17.16, "ce": 0.003694644052302465, "kl": 0.0022393312101485208, "context": 0.004814309650100768}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.23538713157176971, "loss_scale": 4.0, "retries": 0, "seconds": 62.1, "max_memory_gb": 17.16, "ce": 0.011623866783338599, "kl": 0.02566920393383043, "context": 0.0244584682804998}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.6969559192657471, "loss_scale": 4.0, "retries": 0, "seconds": 63.3, "max_memory_gb": 17.16, "ce": 0.059954256896162406, "kl": 0.012717045610770583, "context": 0.06631278138956986}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 3.4543471336364746, "loss_scale": 4.0, "retries": 0, "seconds": 64.7, "max_memory_gb": 17.16, "ce": 0.10677461011800915, "kl": 0.01132123326033252, "context": 0.11243522698350716}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.4304078221321106, "loss_scale": 4.0, "retries": 0, "seconds": 65.9, "max_memory_gb": 17.16, "ce": 0.017047878936864436, "kl": 0.11448673985432833, "context": 0.07429124973714352}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 1.5483053922653198, "loss_scale": 4.0, "retries": 0, "seconds": 67.1, "max_memory_gb": 17.16, "ce": 0.03697559708962217, "kl": 0.2115026672836393, "context": 0.1427269265986979}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 1.627018928527832, "loss_scale": 4.0, "retries": 0, "seconds": 68.4, "max_memory_gb": 17.16, "ce": 0.08369070899789222, "kl": 0.06388846066693077, "context": 0.11563493887661025}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 0.4386686086654663, "loss_scale": 4.0, "retries": 0, "seconds": 69.8, "max_memory_gb": 17.16, "ce": 0.02628110950172413, "kl": 0.01530374323556316, "context": 0.03393298064474948}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.408914715051651, "loss_scale": 4.0, "retries": 0, "seconds": 71.0, "max_memory_gb": 17.16, "ce": 0.0239382884465158, "kl": 0.05526355537585914, "context": 0.051570067182183266}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.10927658528089523, "loss_scale": 4.0, "retries": 0, "seconds": 72.4, "max_memory_gb": 17.16, "ce": 0.005407058837590739, "kl": 0.006270585243328242, "context": 0.008542351628420874}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.014561985619366169, "loss_scale": 4.0, "retries": 0, "seconds": 73.6, "max_memory_gb": 17.16, "ce": 0.0008591730802436359, "kl": 0.0001932194068103854, "context": 0.0009557827943353914}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 1.6307015419006348, "loss_scale": 4.0, "retries": 0, "seconds": 74.9, "max_memory_gb": 17.16, "ce": 0.15102842124179006, "kl": 0.07310676799534122, "context": 0.18758181363227777}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 1.1426122188568115, "loss_scale": 4.0, "retries": 0, "seconds": 76.1, "max_memory_gb": 17.16, "ce": 0.09484874230111018, "kl": 0.06944672168174293, "context": 0.12957210309104994}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 1.3831816911697388, "loss_scale": 4.0, "retries": 0, "seconds": 77.3, "max_memory_gb": 17.16, "ce": 0.1064396562287584, "kl": 0.4433228373163729, "context": 0.32810106888064183}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.542778730392456, "loss_scale": 4.0, "retries": 0, "seconds": 78.6, "max_memory_gb": 17.16, "ce": 0.03664515956188552, "kl": 0.008166574843926355, "context": 0.040728445863351226}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 1.1461325883865356, "loss_scale": 4.0, "retries": 0, "seconds": 79.8, "max_memory_gb": 17.16, "ce": 0.06842581661476288, "kl": 0.11145696675521322, "context": 0.12415429949760437}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.127059206366539, "loss_scale": 4.0, "retries": 0, "seconds": 81.1, "max_memory_gb": 17.16, "ce": 0.007362864329479635, "kl": 0.0039622593321837485, "context": 0.009343994199298322}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 0.7380410432815552, "loss_scale": 4.0, "retries": 0, "seconds": 82.4, "max_memory_gb": 17.16, "ce": 0.1287905341014266, "kl": 0.11851461138576269, "context": 0.18804783979430795}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 1.834454894065857, "loss_scale": 4.0, "retries": 0, "seconds": 83.6, "max_memory_gb": 17.16, "ce": 0.1379174486237389, "kl": 0.06960331992468127, "context": 0.172719104855787}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.07685001194477081, "loss_scale": 4.0, "retries": 0, "seconds": 84.8, "max_memory_gb": 17.16, "ce": 0.0042577920357871335, "kl": 0.005856654035142128, "context": 0.007186118979007006}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.291485995054245, "loss_scale": 4.0, "retries": 0, "seconds": 86.4, "max_memory_gb": 17.16, "ce": 0.010831499803316547, "kl": 0.009142374383372953, "context": 0.015402686629386153}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 3.188506841659546, "loss_scale": 4.0, "retries": 0, "seconds": 88.0, "max_memory_gb": 17.16, "ce": 0.6748900502425386, "kl": 0.00953108584144502, "context": 0.6796555738255847}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 1.5551156997680664, "loss_scale": 4.0, "retries": 0, "seconds": 89.6, "max_memory_gb": 17.16, "ce": 0.016360022738808766, "kl": 0.0072472172305424465, "context": 0.01998363182065077}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.08194348216056824, "loss_scale": 4.0, "retries": 0, "seconds": 90.7, "max_memory_gb": 17.16, "ce": 0.003273289454227779, "kl": 0.005230942588241305, "context": 0.005888760984817054}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 1.1825932264328003, "loss_scale": 4.0, "retries": 0, "seconds": 91.9, "max_memory_gb": 17.16, "ce": 0.007605110280564986, "kl": 0.48459539077884983, "context": 0.24990280229394557}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 2.269217014312744, "loss_scale": 4.0, "retries": 0, "seconds": 93.2, "max_memory_gb": 17.16, "ce": 0.1724174051414593, "kl": 0.03084166138796718, "context": 0.18783823768171715}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.8670785427093506, "loss_scale": 4.0, "retries": 0, "seconds": 94.4, "max_memory_gb": 17.16, "ce": 0.09959488613094436, "kl": 0.11434786291533783, "context": 0.1567688250252104}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 1.9679723978042603, "loss_scale": 4.0, "retries": 0, "seconds": 95.8, "max_memory_gb": 17.16, "ce": 0.5108777233399451, "kl": 0.810406188480556, "context": 0.9160808296874166}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 1.9252161979675293, "loss_scale": 4.0, "retries": 0, "seconds": 97.6, "max_memory_gb": 17.16, "ce": 0.01377915440389188, "kl": 1.324180617928505, "context": 0.6758694488089532}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.9995101094245911, "loss_scale": 4.0, "retries": 0, "seconds": 98.8, "max_memory_gb": 17.16, "ce": 0.015859210747294128, "kl": 0.011840839113574475, "context": 0.021779630333185196}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.47551366686820984, "loss_scale": 4.0, "retries": 0, "seconds": 100.0, "max_memory_gb": 17.16, "ce": 0.002088322493364103, "kl": 0.08295084994460922, "context": 0.043563748738961294}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.48559775948524475, "loss_scale": 4.0, "retries": 0, "seconds": 101.3, "max_memory_gb": 17.16, "ce": 0.01278143044328317, "kl": 0.01361057793837972, "context": 0.01958671941247303}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.3474651873111725, "loss_scale": 4.0, "retries": 0, "seconds": 102.5, "max_memory_gb": 17.16, "ce": 0.0019474766668281518, "kl": 0.048188366899921675, "context": 0.026041659533802886}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 0.9853038787841797, "loss_scale": 4.0, "retries": 0, "seconds": 103.8, "max_memory_gb": 17.16, "ce": 0.11126299307215959, "kl": 0.03555402059282642, "context": 0.1290400021825917}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.2476140409708023, "loss_scale": 4.0, "retries": 0, "seconds": 105.0, "max_memory_gb": 17.16, "ce": 0.013347955129574984, "kl": 0.011300544021651149, "context": 0.018998226849362254}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 1.711542010307312, "loss_scale": 4.0, "retries": 0, "seconds": 106.2, "max_memory_gb": 17.16, "ce": 0.12670466605050024, "kl": 0.5804431886572274, "context": 0.41692625409632456}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 1.2317031621932983, "loss_scale": 4.0, "retries": 0, "seconds": 107.5, "max_memory_gb": 17.16, "ce": 0.0683220089704264, "kl": 0.03274848894443494, "context": 0.08469625491125043}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.11808847635984421, "loss_scale": 4.0, "retries": 0, "seconds": 108.7, "max_memory_gb": 17.16, "ce": 0.003160193078656448, "kl": 0.0004843633441034001, "context": 0.003402374728466384}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 1.494277000427246, "loss_scale": 4.0, "retries": 0, "seconds": 109.9, "max_memory_gb": 17.16, "ce": 0.04790459533978719, "kl": 0.19920438090048265, "context": 0.1475067816209048}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 1.7224990129470825, "loss_scale": 4.0, "retries": 0, "seconds": 111.3, "max_memory_gb": 17.16, "ce": 0.0068828644580207765, "kl": 0.5330781454103999, "context": 0.2734219296835363}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.3597259819507599, "loss_scale": 4.0, "retries": 0, "seconds": 112.5, "max_memory_gb": 17.16, "ce": 0.017151457926956937, "kl": 0.0035452316318469457, "context": 0.01892407421109965}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.3946971893310547, "loss_scale": 4.0, "retries": 0, "seconds": 113.7, "max_memory_gb": 17.16, "ce": 0.03169777379662264, "kl": 0.01771819707209943, "context": 0.04055687128857244}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.4633689820766449, "loss_scale": 4.0, "retries": 0, "seconds": 115.0, "max_memory_gb": 17.16, "ce": 0.014206007675966248, "kl": 0.004015640311990865, "context": 0.01621382834855467}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 2.7626569271087646, "loss_scale": 4.0, "retries": 0, "seconds": 116.2, "max_memory_gb": 17.16, "ce": 0.3214096666779369, "kl": 0.30601819399453234, "context": 0.4744187608885113}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 4.94952917098999, "loss_scale": 4.0, "retries": 0, "seconds": 117.5, "max_memory_gb": 17.16, "ce": 0.1881268275756156, "kl": 0.27152498505893163, "context": 0.3238893124216702}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 3.7212657928466797, "loss_scale": 4.0, "retries": 0, "seconds": 118.7, "max_memory_gb": 17.16, "ce": 0.07070442621625261, "kl": 0.2982696667231721, "context": 0.21983925148379058}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 1.3265507221221924, "loss_scale": 4.0, "retries": 0, "seconds": 119.9, "max_memory_gb": 17.16, "ce": 0.04902962519554421, "kl": 0.01036001171451062, "context": 0.0542096329445485}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 3.6732728481292725, "loss_scale": 4.0, "retries": 0, "seconds": 121.2, "max_memory_gb": 17.16, "ce": 0.5187553424511862, "kl": 0.013151904247933999, "context": 0.5253312959102914}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 2.8632616996765137, "loss_scale": 4.0, "retries": 0, "seconds": 122.4, "max_memory_gb": 17.16, "ce": 0.3272478231738205, "kl": 0.005284036891680444, "context": 0.32988985112751834}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.44038042426109314, "loss_scale": 4.0, "retries": 0, "seconds": 123.6, "max_memory_gb": 17.16, "ce": 0.015556237543933094, "kl": 0.020221004204358906, "context": 0.02566673932597041}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.4064614474773407, "loss_scale": 4.0, "retries": 0, "seconds": 125.0, "max_memory_gb": 17.16, "ce": 0.0019123016463709064, "kl": 0.09017590263283637, "context": 0.047000251681311056}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.16115118563175201, "loss_scale": 4.0, "retries": 0, "seconds": 126.3, "max_memory_gb": 17.16, "ce": 0.01756212995678652, "kl": 0.007296416490135016, "context": 0.02121033771254588}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 1.0858324766159058, "loss_scale": 4.0, "retries": 0, "seconds": 127.5, "max_memory_gb": 17.16, "ce": 0.05389453509997111, "kl": 0.36060524776621605, "context": 0.2341971626447048}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.20024091005325317, "loss_scale": 4.0, "retries": 0, "seconds": 128.8, "max_memory_gb": 17.16, "ce": 0.00534465562668629, "kl": 0.020567334489896894, "context": 0.01562832249328494}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.32668620347976685, "loss_scale": 4.0, "retries": 0, "seconds": 130.0, "max_memory_gb": 17.16, "ce": 0.02723279205383733, "kl": 0.008913932462746743, "context": 0.03168975800508633}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.1584233045578003, "loss_scale": 4.0, "retries": 0, "seconds": 131.3, "max_memory_gb": 17.16, "ce": 0.010677547426894307, "kl": 0.004257306121871807, "context": 0.012806200596969575}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.14521516859531403, "loss_scale": 4.0, "retries": 0, "seconds": 132.5, "max_memory_gb": 17.16, "ce": 0.01070091436849907, "kl": 0.00336813737931152, "context": 0.01238498359452933}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 0.7990106344223022, "loss_scale": 4.0, "retries": 0, "seconds": 133.7, "max_memory_gb": 17.16, "ce": 0.022341412884998135, "kl": 0.3567986059933901, "context": 0.20074072241550311}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.02860288880765438, "loss_scale": 4.0, "retries": 0, "seconds": 135.0, "max_memory_gb": 17.16, "ce": 0.002541441543144174, "kl": 0.00036368129804031923, "context": 0.00272328220307827}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.5165815353393555, "loss_scale": 4.0, "retries": 0, "seconds": 136.2, "max_memory_gb": 17.16, "ce": 0.03831499917578185, "kl": 0.15331045106086094, "context": 0.1149702210968826}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 2.3704254627227783, "loss_scale": 4.0, "retries": 0, "seconds": 137.4, "max_memory_gb": 17.16, "ce": 0.30670362412638497, "kl": 0.0029032834474946867, "context": 0.3081552668882068}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 3.20835542678833, "loss_scale": 4.0, "retries": 0, "seconds": 138.7, "max_memory_gb": 17.16, "ce": 0.1668098312075017, "kl": 0.14132825890555978, "context": 0.23747395747341216}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.46098682284355164, "loss_scale": 4.0, "retries": 0, "seconds": 139.9, "max_memory_gb": 17.16, "ce": 0.034814716782420874, "kl": 0.0056052896252367646, "context": 0.03761736233718693}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.023196827620267868, "loss_scale": 4.0, "retries": 0, "seconds": 141.1, "max_memory_gb": 17.16, "ce": 0.0016973337333183736, "kl": 0.0007011723791947588, "context": 0.0020479199301917106}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.27350762486457825, "loss_scale": 4.0, "retries": 0, "seconds": 142.5, "max_memory_gb": 17.16, "ce": 0.022188722272403538, "kl": 0.0031655190632591257, "context": 0.023771480831783265}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 0.9556149840354919, "loss_scale": 4.0, "retries": 0, "seconds": 143.7, "max_memory_gb": 17.16, "ce": 0.009877861535642296, "kl": 0.08129274354723748, "context": 0.05052423244342208}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 2.113013505935669, "loss_scale": 4.0, "retries": 0, "seconds": 145.0, "max_memory_gb": 17.16, "ce": 0.020291872609959682, "kl": 0.3540237630240881, "context": 0.19730375226208707}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 1.2463531494140625, "loss_scale": 4.0, "retries": 0, "seconds": 146.2, "max_memory_gb": 17.16, "ce": 0.04026876893476583, "kl": 0.0861666223499924, "context": 0.08335207821801305}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 0.8171836137771606, "loss_scale": 4.0, "retries": 0, "seconds": 147.4, "max_memory_gb": 17.16, "ce": 0.09373606517328881, "kl": 0.29195785884985526, "context": 0.23971499459003098}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.09874286502599716, "loss_scale": 4.0, "retries": 0, "seconds": 148.8, "max_memory_gb": 17.16, "ce": 0.006433116432162933, "kl": 0.00556802237588272, "context": 0.009217127386364155}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.3344944715499878, "loss_scale": 4.0, "retries": 0, "seconds": 149.9, "max_memory_gb": 17.16, "ce": 0.21353396214544773, "kl": 0.001884145973235718, "context": 0.21447604210698046}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.1157192662358284, "loss_scale": 4.0, "retries": 0, "seconds": 151.1, "max_memory_gb": 17.16, "ce": 0.00562883292150218, "kl": 0.0038191882772480312, "context": 0.007538427191320807}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 3.3750665187835693, "loss_scale": 4.0, "retries": 0, "seconds": 152.5, "max_memory_gb": 17.16, "ce": 0.20169349809293635, "kl": 0.220997154712677, "context": 0.31219207265530713}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 0.6778520941734314, "loss_scale": 4.0, "retries": 0, "seconds": 153.7, "max_memory_gb": 17.16, "ce": 0.052496150849037804, "kl": 0.03623905184940668, "context": 0.0706156748929061}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.103799819946289, "loss_scale": 4.0, "retries": 0, "seconds": 155.0, "max_memory_gb": 17.16, "ce": 0.141373484118958, "kl": 0.030092535744188353, "context": 0.15641975443577394}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 0.5075649619102478, "loss_scale": 4.0, "retries": 0, "seconds": 156.3, "max_memory_gb": 17.16, "ce": 0.0454917405731976, "kl": 0.02545213609118946, "context": 0.05821780697442591}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 3.8108155727386475, "loss_scale": 4.0, "retries": 0, "seconds": 157.5, "max_memory_gb": 17.16, "ce": 0.6138681942320545, "kl": 0.433790178491563, "context": 0.8307632853393443}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 0.9551974534988403, "loss_scale": 4.0, "retries": 0, "seconds": 158.8, "max_memory_gb": 17.16, "ce": 0.12533350093872286, "kl": 0.07495683394290609, "context": 0.16281191798043437}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 4.406610488891602, "loss_scale": 4.0, "retries": 0, "seconds": 160.0, "max_memory_gb": 17.16, "ce": 0.7245136136189103, "kl": 0.10493356920778751, "context": 0.776980409398675}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.09210564196109772, "loss_scale": 4.0, "retries": 0, "seconds": 161.2, "max_memory_gb": 17.16, "ce": 0.006919187537278049, "kl": 0.0007240800116505852, "context": 0.007281227502971888}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 2.7002501487731934, "loss_scale": 4.0, "retries": 0, "seconds": 162.5, "max_memory_gb": 17.16, "ce": 0.12660439964747638, "kl": 0.4554487828136189, "context": 0.3543287823558785}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 0.8117956519126892, "loss_scale": 4.0, "retries": 0, "seconds": 163.7, "max_memory_gb": 17.16, "ce": 0.08668227141606621, "kl": 0.038261366044025635, "context": 0.1058129542216193}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.6317869424819946, "loss_scale": 4.0, "retries": 0, "seconds": 164.9, "max_memory_gb": 17.16, "ce": 0.028036460164003074, "kl": 0.2675335555541096, "context": 0.1618032359983772}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 4.925462245941162, "loss_scale": 4.0, "retries": 0, "seconds": 166.2, "max_memory_gb": 17.16, "ce": 0.3287813103524968, "kl": 0.3722318405925762, "context": 0.5148972438182682}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 1.0855774879455566, "loss_scale": 4.0, "retries": 0, "seconds": 167.4, "max_memory_gb": 17.16, "ce": 0.1890853360237088, "kl": 0.046253394342784304, "context": 0.21221203508321196}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 1.2791883945465088, "loss_scale": 4.0, "retries": 0, "seconds": 168.6, "max_memory_gb": 17.16, "ce": 0.31413492032515933, "kl": 0.02046965467161499, "context": 0.3243697569705546}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 1.1054524183273315, "loss_scale": 4.0, "retries": 0, "seconds": 170.0, "max_memory_gb": 17.16, "ce": 0.18259471259079874, "kl": 0.004811412800336257, "context": 0.18500041813240387}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 0.4505503177642822, "loss_scale": 4.0, "retries": 0, "seconds": 171.2, "max_memory_gb": 17.17, "ce": 0.013921618490712717, "kl": 0.050111156477214536, "context": 0.0389771971385926}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 2.71380615234375, "loss_scale": 4.0, "retries": 0, "seconds": 172.7, "max_memory_gb": 17.17, "ce": 0.05026786250527948, "kl": 0.10363782667172927, "context": 0.10208677593618631}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 0.4427587389945984, "loss_scale": 4.0, "retries": 0, "seconds": 173.9, "max_memory_gb": 17.17, "ce": 0.015634412025974598, "kl": 0.024828169134707423, "context": 0.028048497493728064}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 3.291167974472046, "loss_scale": 4.0, "retries": 0, "seconds": 175.1, "max_memory_gb": 17.17, "ce": 0.10247478308156133, "kl": 0.15962863946333528, "context": 0.1822891067713499}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 1.0842549800872803, "loss_scale": 4.0, "retries": 0, "seconds": 176.4, "max_memory_gb": 17.17, "ce": 0.013044425315456465, "kl": 0.10117432808488047, "context": 0.06363158894964727}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.09785107523202896, "loss_scale": 4.0, "retries": 0, "seconds": 177.7, "max_memory_gb": 17.17, "ce": 0.00631313567282632, "kl": 0.002391756805081968, "context": 0.007509013841627166}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.08658788353204727, "loss_scale": 4.0, "retries": 0, "seconds": 178.8, "max_memory_gb": 17.17, "ce": 0.009263070773158688, "kl": 0.0007870801819080953, "context": 0.009656610614911187}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 2.3281779289245605, "loss_scale": 4.0, "retries": 0, "seconds": 180.1, "max_memory_gb": 17.17, "ce": 0.10191467148251832, "kl": 0.02240631949280214, "context": 0.11311783036217093}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 3.1510777473449707, "loss_scale": 4.0, "retries": 0, "seconds": 181.3, "max_memory_gb": 17.17, "ce": 0.2603561933792662, "kl": 0.030480052748316666, "context": 0.27559622831176966}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 2.0876059532165527, "loss_scale": 4.0, "retries": 0, "seconds": 182.5, "max_memory_gb": 17.17, "ce": 0.037930753896944225, "kl": 0.11751521460246295, "context": 0.09668836306082085}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.027415672317147255, "loss_scale": 4.0, "retries": 0, "seconds": 183.8, "max_memory_gb": 17.17, "ce": 0.001454884113627486, "kl": 0.002149615154849016, "context": 0.002529691773816012}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.17043885588645935, "loss_scale": 4.0, "retries": 0, "seconds": 185.0, "max_memory_gb": 17.17, "ce": 0.017245425668079406, "kl": 0.009967090278223623, "context": 0.02222897052706685}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 1.822371482849121, "loss_scale": 4.0, "retries": 0, "seconds": 186.4, "max_memory_gb": 17.17, "ce": 0.3577349697443424, "kl": 0.39730709642753936, "context": 0.5563885137089528}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.23307597637176514, "loss_scale": 4.0, "retries": 0, "seconds": 187.6, "max_memory_gb": 17.17, "ce": 0.013679720454092603, "kl": 0.010664377201464958, "context": 0.01901190952048637}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 0.17888282239437103, "loss_scale": 4.0, "retries": 0, "seconds": 188.7, "max_memory_gb": 17.17, "ce": 0.016652911028359085, "kl": 0.002173054268496344, "context": 0.017739437811542302}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.6144518852233887, "loss_scale": 4.0, "retries": 0, "seconds": 190.1, "max_memory_gb": 17.17, "ce": 0.062030863107793266, "kl": 0.03091416640654643, "context": 0.07748794812869164}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 0.5390726923942566, "loss_scale": 4.0, "retries": 0, "seconds": 191.3, "max_memory_gb": 17.17, "ce": 0.024851076625054702, "kl": 0.030754381929000374, "context": 0.040228267665952444}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 0.9033920168876648, "loss_scale": 4.0, "retries": 0, "seconds": 192.6, "max_memory_gb": 17.17, "ce": 0.003193859083694406, "kl": 0.12716096935218957, "context": 0.06677434511948377}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.8130332827568054, "loss_scale": 4.0, "retries": 0, "seconds": 194.0, "max_memory_gb": 17.17, "ce": 0.004708243417553604, "kl": 0.45045332217705436, "context": 0.22993490172666498}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.5051224231719971, "loss_scale": 4.0, "retries": 0, "seconds": 195.2, "max_memory_gb": 17.17, "ce": 0.043819483937113546, "kl": 0.09729210475052241, "context": 0.0924655328199151}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 1.3609585762023926, "loss_scale": 4.0, "retries": 0, "seconds": 196.4, "max_memory_gb": 17.17, "ce": 0.2312837261124514, "kl": 0.0113803571639437, "context": 0.23697390277811792}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 1.7155885696411133, "loss_scale": 4.0, "retries": 0, "seconds": 197.7, "max_memory_gb": 17.17, "ce": 0.026462794063263573, "kl": 0.015546517822258465, "context": 0.03423605294665322}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.30640852451324463, "loss_scale": 4.0, "retries": 0, "seconds": 199.0, "max_memory_gb": 17.17, "ce": 0.025926443748176098, "kl": 0.027389904571464285, "context": 0.03962139482609928}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 1.205410122871399, "loss_scale": 4.0, "retries": 0, "seconds": 200.3, "max_memory_gb": 17.17, "ce": 0.13784573040902615, "kl": 0.11228147841029568, "context": 0.19398646848276258}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 0.6922334432601929, "loss_scale": 4.0, "retries": 0, "seconds": 201.5, "max_memory_gb": 17.17, "ce": 0.0076946099870838225, "kl": 0.037436859114677645, "context": 0.026413038605824113}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 1.2475457191467285, "loss_scale": 4.0, "retries": 0, "seconds": 202.8, "max_memory_gb": 17.17, "ce": 0.05914726064656861, "kl": 0.216420271410243, "context": 0.16735739685827866}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 1.3025096654891968, "loss_scale": 4.0, "retries": 0, "seconds": 204.1, "max_memory_gb": 17.17, "ce": 0.143452717049513, "kl": 0.13658134175057057, "context": 0.21174339164281264}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 1.9557996988296509, "loss_scale": 4.0, "retries": 0, "seconds": 205.3, "max_memory_gb": 17.17, "ce": 0.09616139568970539, "kl": 0.1221874192397081, "context": 0.15725510535412468}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 6.919973850250244, "loss_scale": 4.0, "retries": 0, "seconds": 206.5, "max_memory_gb": 17.17, "ce": 0.2499775364267407, "kl": 0.6758843814241118, "context": 0.5879197235335596}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 4.072597026824951, "loss_scale": 4.0, "retries": 0, "seconds": 207.8, "max_memory_gb": 17.17, "ce": 0.21462032751878724, "kl": 0.19918440133915283, "context": 0.31421252818836365}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.20770925283432007, "loss_scale": 4.0, "retries": 0, "seconds": 209.0, "max_memory_gb": 17.17, "ce": 0.012694863093202002, "kl": 0.008882745038135909, "context": 0.01713623502291739}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 15.341867446899414, "loss_scale": 4.0, "retries": 0, "seconds": 210.2, "max_memory_gb": 17.17, "ce": 0.36550182348582894, "kl": 0.06510215868183877, "context": 0.39805291302036494}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.876274049282074, "loss_scale": 4.0, "retries": 0, "seconds": 211.5, "max_memory_gb": 17.17, "ce": 0.03980392322409898, "kl": 0.007856400894524995, "context": 0.043732124599046074}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.1447637528181076, "loss_scale": 4.0, "retries": 0, "seconds": 212.7, "max_memory_gb": 17.17, "ce": 0.013575219265476335, "kl": 0.0023093652416719124, "context": 0.014729901973623782}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 1.6495522260665894, "loss_scale": 4.0, "retries": 0, "seconds": 214.0, "max_memory_gb": 17.17, "ce": 0.06555184900935274, "kl": 0.8996457122266293, "context": 0.5153747200965881}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.01812422275543213, "loss_scale": 4.0, "retries": 0, "seconds": 215.3, "max_memory_gb": 17.17, "ce": 0.0010146442291443236, "kl": 8.383407430301304e-05, "context": 0.0010565612683421932}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.645767867565155, "loss_scale": 4.0, "retries": 0, "seconds": 216.5, "max_memory_gb": 17.17, "ce": 0.015791044133948162, "kl": 0.05565697094425559, "context": 0.043619528878480196}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.15539363026618958, "loss_scale": 4.0, "retries": 0, "seconds": 217.8, "max_memory_gb": 17.17, "ce": 0.008921771863242611, "kl": 0.011798248880950268, "context": 0.014820896089076996}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.15484082698822021, "loss_scale": 4.0, "retries": 0, "seconds": 219.0, "max_memory_gb": 17.17, "ce": 0.010530881816521287, "kl": 0.002689408720470965, "context": 0.01187558617675677}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.44153454899787903, "loss_scale": 4.0, "retries": 0, "seconds": 220.2, "max_memory_gb": 17.17, "ce": 0.03685270399728324, "kl": 0.009790663834507995, "context": 0.041748035451746546}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 0.6755359172821045, "loss_scale": 4.0, "retries": 0, "seconds": 221.5, "max_memory_gb": 17.17, "ce": 0.07102561456849799, "kl": 0.03972501365933567, "context": 0.09088812256231904}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.06968773901462555, "loss_scale": 4.0, "retries": 0, "seconds": 222.7, "max_memory_gb": 17.17, "ce": 0.0047513749086647294, "kl": 0.0010061432685688487, "context": 0.0052544463032973}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 0.9946143627166748, "loss_scale": 4.0, "retries": 0, "seconds": 223.9, "max_memory_gb": 17.17, "ce": 0.035072689614025876, "kl": 0.6590079011602938, "context": 0.36457663180772215}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 2.3622548580169678, "loss_scale": 4.0, "retries": 0, "seconds": 225.2, "max_memory_gb": 17.17, "ce": 0.17842643207404763, "kl": 0.008221010764827952, "context": 0.1825369440484792}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.1258623600006104, "loss_scale": 4.0, "retries": 0, "seconds": 226.4, "max_memory_gb": 17.17, "ce": 0.30863573440001346, "kl": 0.009191210971039254, "context": 0.3132313310634345}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.8550876975059509, "loss_scale": 4.0, "retries": 0, "seconds": 227.7, "max_memory_gb": 17.17, "ce": 0.045094317974871956, "kl": 0.035656137551995926, "context": 0.06292238860623911}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 0.5341698527336121, "loss_scale": 4.0, "retries": 0, "seconds": 229.0, "max_memory_gb": 17.17, "ce": 0.0028195093327667564, "kl": 0.10549018673191313, "context": 0.05556460120715201}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 0.8536122441291809, "loss_scale": 4.0, "retries": 0, "seconds": 230.2, "max_memory_gb": 17.17, "ce": 0.06822399969678372, "kl": 0.19798585060016194, "context": 0.16721692105056718}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.038656510412693024, "loss_scale": 4.0, "retries": 0, "seconds": 231.5, "max_memory_gb": 17.17, "ce": 0.002690575201995671, "kl": 0.00021803068011649884, "context": 0.0027995905693387613}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.08955885469913483, "loss_scale": 4.0, "retries": 0, "seconds": 232.7, "max_memory_gb": 17.17, "ce": 0.0029995213844813406, "kl": 0.003151787332171807, "context": 0.004575414946884848}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.3883706033229828, "loss_scale": 4.0, "retries": 0, "seconds": 233.9, "max_memory_gb": 17.17, "ce": 0.006264432326133829, "kl": 0.09123972087309085, "context": 0.051884291067835875}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 5.209321975708008, "loss_scale": 4.0, "retries": 0, "seconds": 235.2, "max_memory_gb": 17.17, "ce": 0.06894877518061548, "kl": 0.10238297926844098, "context": 0.12014026464021299}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 0.7805030941963196, "loss_scale": 4.0, "retries": 0, "seconds": 236.4, "max_memory_gb": 17.17, "ce": 0.017303062370046973, "kl": 0.615225313915289, "context": 0.32491572913568234}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.3698372840881348, "loss_scale": 4.0, "retries": 0, "seconds": 237.6, "max_memory_gb": 17.17, "ce": 0.20404100505402312, "kl": 0.008666738510328287, "context": 0.2083743747934932}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 4.082984924316406, "loss_scale": 4.0, "retries": 0, "seconds": 238.9, "max_memory_gb": 17.17, "ce": 0.3407572863070527, "kl": 0.20233890791860176, "context": 0.4419267269986449}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.03297072649002075, "loss_scale": 4.0, "retries": 0, "seconds": 240.1, "max_memory_gb": 17.17, "ce": 0.0011124551383545622, "kl": 0.0022106791075202636, "context": 0.002217794710304588}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 5.224308013916016, "loss_scale": 4.0, "retries": 0, "seconds": 241.4, "max_memory_gb": 17.17, "ce": 0.20167990843765438, "kl": 0.15097874788079935, "context": 0.2771692896494642}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.318860411643982, "loss_scale": 4.0, "retries": 0, "seconds": 242.6, "max_memory_gb": 17.17, "ce": 0.39634535522782244, "kl": 0.02448938808447565, "context": 0.40859003610967193}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.05437874048948288, "loss_scale": 4.0, "retries": 0, "seconds": 243.8, "max_memory_gb": 17.17, "ce": 0.0029274143162183464, "kl": 0.0019677072150443564, "context": 0.003911267849616706}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 3.1011950969696045, "loss_scale": 4.0, "retries": 0, "seconds": 245.2, "max_memory_gb": 17.17, "ce": 0.12218646955443546, "kl": 0.0033989049334195442, "context": 0.12388592585921288}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.10770358145236969, "loss_scale": 4.0, "retries": 0, "seconds": 246.4, "max_memory_gb": 17.17, "ce": 0.011469749544630758, "kl": 0.0008599499378760811, "context": 0.011899724850081839}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 1.0590163469314575, "loss_scale": 4.0, "retries": 0, "seconds": 247.6, "max_memory_gb": 17.17, "ce": 0.0033244705555262044, "kl": 0.23651587432266297, "context": 0.12158240507051232}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 5.148157119750977, "loss_scale": 4.0, "retries": 0, "seconds": 248.9, "max_memory_gb": 17.17, "ce": 0.023786629746609833, "kl": 0.16514145575570183, "context": 0.10635735721734818}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.7821419835090637, "loss_scale": 4.0, "retries": 0, "seconds": 250.1, "max_memory_gb": 17.17, "ce": 0.08131699714431306, "kl": 0.010511009433685103, "context": 0.0865725011244649}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.2360705137252808, "loss_scale": 4.0, "retries": 0, "seconds": 251.3, "max_memory_gb": 17.17, "ce": 0.13565101905260235, "kl": 0.008819892496831017, "context": 0.14006096762022935}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.185009241104126, "loss_scale": 4.0, "retries": 0, "seconds": 252.6, "max_memory_gb": 17.17, "ce": 0.24585758452303708, "kl": 0.02540702458281885, "context": 0.2585611001122743}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 1.3963193893432617, "loss_scale": 4.0, "retries": 0, "seconds": 253.9, "max_memory_gb": 17.17, "ce": 0.06777095021971036, "kl": 0.0728801420518721, "context": 0.1042110251437407}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.06919928640127182, "loss_scale": 4.0, "retries": 0, "seconds": 255.2, "max_memory_gb": 17.17, "ce": 0.004172720335191116, "kl": 0.002394182346733942, "context": 0.005369811522541568}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.9566669464111328, "loss_scale": 4.0, "retries": 0, "seconds": 256.3, "max_memory_gb": 17.17, "ce": 0.0274232203628344, "kl": 0.028666407175478525, "context": 0.0417564240815409}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.42783209681510925, "loss_scale": 4.0, "retries": 0, "seconds": 257.5, "max_memory_gb": 17.17, "ce": 0.039185192494187504, "kl": 0.0038378986646421254, "context": 0.04110414127353579}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 1.133622646331787, "loss_scale": 4.0, "retries": 0, "seconds": 258.8, "max_memory_gb": 17.17, "ce": 0.005616052687400952, "kl": 0.3719773590564728, "context": 0.1916047351260204}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.10328634083271027, "loss_scale": 4.0, "retries": 0, "seconds": 260.0, "max_memory_gb": 17.17, "ce": 0.005815956203150563, "kl": 0.0025649447889009025, "context": 0.007098428497556597}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.5706245303153992, "loss_scale": 4.0, "retries": 0, "seconds": 261.2, "max_memory_gb": 17.17, "ce": 0.08383652358315885, "kl": 0.05379634228302166, "context": 0.11073469289112836}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.052644114941358566, "loss_scale": 4.0, "retries": 0, "seconds": 262.6, "max_memory_gb": 17.17, "ce": 0.003358388232300058, "kl": 0.0010868922690860927, "context": 0.003901834395946935}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 1.9831444025039673, "loss_scale": 4.0, "retries": 0, "seconds": 263.8, "max_memory_gb": 17.17, "ce": 0.2187887804757338, "kl": 0.04921949643176049, "context": 0.24339853436686099}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 4.983897686004639, "loss_scale": 4.0, "retries": 0, "seconds": 265.0, "max_memory_gb": 17.17, "ce": 0.3996246985043399, "kl": 0.08962183331823326, "context": 0.4444356075109681}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.6464053988456726, "loss_scale": 4.0, "retries": 0, "seconds": 266.4, "max_memory_gb": 17.17, "ce": 0.12686570070218295, "kl": 0.042633273609681055, "context": 0.14818233891855925}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 8.310133934020996, "loss_scale": 4.0, "retries": 0, "seconds": 267.6, "max_memory_gb": 17.17, "ce": 0.1273348938266281, "kl": 0.9797830650422839, "context": 0.6172264292836189}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.8636290431022644, "loss_scale": 4.0, "retries": 0, "seconds": 269.0, "max_memory_gb": 17.17, "ce": 0.19002575638296548, "kl": 0.022914517332537798, "context": 0.20148301706649363}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.5315390825271606, "loss_scale": 4.0, "retries": 0, "seconds": 270.2, "max_memory_gb": 17.17, "ce": 0.29074179245799314, "kl": 0.06807014308287762, "context": 0.3247768533183262}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 2.702221155166626, "loss_scale": 4.0, "retries": 0, "seconds": 271.4, "max_memory_gb": 17.17, "ce": 0.5090759836602956, "kl": 0.021376083388531697, "context": 0.5197640262776986}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.650112509727478, "loss_scale": 4.0, "retries": 0, "seconds": 272.7, "max_memory_gb": 17.17, "ce": 0.029736984113696963, "kl": 0.08236766131813056, "context": 0.0709208152256906}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.05542221665382385, "loss_scale": 4.0, "retries": 0, "seconds": 273.9, "max_memory_gb": 17.17, "ce": 0.0035295043635414913, "kl": 0.0018250421987886511, "context": 0.004442025441676378}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 2.150155782699585, "loss_scale": 4.0, "retries": 0, "seconds": 275.2, "max_memory_gb": 17.17, "ce": 0.22215939292800613, "kl": 0.18701323587447405, "context": 0.3156660124659538}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 1.2053349018096924, "loss_scale": 4.0, "retries": 0, "seconds": 276.5, "max_memory_gb": 17.17, "ce": 0.010803923127241433, "kl": 0.2072433161083609, "context": 0.11442558391718194}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 0.24847716093063354, "loss_scale": 4.0, "retries": 0, "seconds": 277.7, "max_memory_gb": 17.17, "ce": 0.025894286227412522, "kl": 0.0024200844763981877, "context": 0.027104328473797068}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.7387921214103699, "loss_scale": 4.0, "retries": 0, "seconds": 278.9, "max_memory_gb": 17.17, "ce": 0.0029487757419701666, "kl": 0.10618491034801991, "context": 0.05604123153898399}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 0.6493692398071289, "loss_scale": 4.0, "retries": 0, "seconds": 280.2, "max_memory_gb": 17.17, "ce": 0.03276075568282977, "kl": 0.054515295023520594, "context": 0.06001840202952735}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 2.6442222595214844, "loss_scale": 4.0, "retries": 0, "seconds": 281.4, "max_memory_gb": 17.17, "ce": 0.14035907652578317, "kl": 0.008970575507191825, "context": 0.14484436385100707}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 0.8011026382446289, "loss_scale": 4.0, "retries": 0, "seconds": 282.8, "max_memory_gb": 17.17, "ce": 0.03392567968694493, "kl": 0.03103837244998431, "context": 0.04944486636668444}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.15902596712112427, "loss_scale": 4.0, "retries": 0, "seconds": 284.0, "max_memory_gb": 17.17, "ce": 0.005209396898862906, "kl": 0.014707851070852485, "context": 0.012563322321511805}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.4207422733306885, "loss_scale": 4.0, "retries": 0, "seconds": 285.2, "max_memory_gb": 17.17, "ce": 0.02467419304593932, "kl": 0.005988139759210753, "context": 0.0276682630792493}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.4345734119415283, "loss_scale": 4.0, "retries": 0, "seconds": 286.5, "max_memory_gb": 17.17, "ce": 0.01470967945351731, "kl": 0.01113280663594196, "context": 0.020276081850170158}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.3372844159603119, "loss_scale": 4.0, "retries": 0, "seconds": 287.7, "max_memory_gb": 17.17, "ce": 0.0077217871039465535, "kl": 0.04143814876442775, "context": 0.02844086172990501}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 6.092164993286133, "loss_scale": 4.0, "retries": 0, "seconds": 288.9, "max_memory_gb": 17.17, "ce": 0.11022170032083523, "kl": 0.09618389427805596, "context": 0.1583136514236685}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 2.420140504837036, "loss_scale": 4.0, "retries": 0, "seconds": 290.2, "max_memory_gb": 17.17, "ce": 0.4594978227687534, "kl": 0.21257982612587512, "context": 0.5657877278281376}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.9723140597343445, "loss_scale": 4.0, "retries": 0, "seconds": 291.4, "max_memory_gb": 17.17, "ce": 0.04462161936680786, "kl": 0.09757363265089225, "context": 0.0934084365144372}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.29659438133239746, "loss_scale": 4.0, "retries": 0, "seconds": 292.6, "max_memory_gb": 17.17, "ce": 0.023009280674159527, "kl": 0.005281581696181092, "context": 0.0256500716204755}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.7551121711730957, "loss_scale": 4.0, "retries": 0, "seconds": 293.9, "max_memory_gb": 17.17, "ce": 0.02499201468890533, "kl": 0.09186436387244612, "context": 0.07092419522814453}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.46637433767318726, "loss_scale": 4.0, "retries": 0, "seconds": 295.1, "max_memory_gb": 17.17, "ce": 0.021135843824595213, "kl": 0.009493121930063353, "context": 0.025882404763251543}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 2.5550591945648193, "loss_scale": 4.0, "retries": 0, "seconds": 296.4, "max_memory_gb": 17.17, "ce": 0.29750214552041143, "kl": 0.032309127011103556, "context": 0.31365672033280134}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.18178167939186096, "loss_scale": 4.0, "retries": 0, "seconds": 297.5, "max_memory_gb": 17.17, "ce": 0.01582889709970914, "kl": 0.01719049844541587, "context": 0.024424146569799632}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 0.9009931683540344, "loss_scale": 4.0, "retries": 0, "seconds": 298.7, "max_memory_gb": 17.17, "ce": 0.05891449593764264, "kl": 0.023611333333064977, "context": 0.07072015981248114}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.2954919636249542, "loss_scale": 4.0, "retries": 0, "seconds": 300.0, "max_memory_gb": 17.17, "ce": 0.019057069097470958, "kl": 0.0035374524486542214, "context": 0.020825795392738655}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.5230159759521484, "loss_scale": 4.0, "retries": 0, "seconds": 301.2, "max_memory_gb": 17.17, "ce": 0.1154683434870094, "kl": 0.016892691475732136, "context": 0.12391468556597829}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 0.28546851873397827, "loss_scale": 4.0, "retries": 0, "seconds": 302.3, "max_memory_gb": 17.17, "ce": 0.02270672406302765, "kl": 0.035116639295665664, "context": 0.04026504361536354}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 1.3120239973068237, "loss_scale": 4.0, "retries": 0, "seconds": 303.7, "max_memory_gb": 17.17, "ce": 0.07225339571596123, "kl": 0.050876188557595015, "context": 0.09769148955820128}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.18144935369491577, "loss_scale": 4.0, "retries": 0, "seconds": 304.9, "max_memory_gb": 17.17, "ce": 0.012358391977613792, "kl": 0.01317463093346305, "context": 0.018945707372040488}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.059926990419626236, "loss_scale": 4.0, "retries": 0, "seconds": 306.1, "max_memory_gb": 17.17, "ce": 0.003948654746636748, "kl": 0.001330305363808293, "context": 0.004613807512214407}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 3.124196767807007, "loss_scale": 4.0, "retries": 0, "seconds": 307.4, "max_memory_gb": 17.17, "ce": 0.02803918606150546, "kl": 0.22666281932561105, "context": 0.14137059828499332}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 2.3647913932800293, "loss_scale": 4.0, "retries": 0, "seconds": 308.6, "max_memory_gb": 17.17, "ce": 0.2783568673330592, "kl": 0.04500521146110259, "context": 0.30085947454790585}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.027307648211717606, "loss_scale": 4.0, "retries": 0, "seconds": 309.9, "max_memory_gb": 17.17, "ce": 0.0017183727359224577, "kl": 0.0004957717269462592, "context": 0.0019662586564663798}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 1.258786678314209, "loss_scale": 4.0, "retries": 0, "seconds": 311.1, "max_memory_gb": 17.17, "ce": 0.010808858191012405, "kl": 0.21998671814981208, "context": 0.12080222069926094}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.21841934323310852, "loss_scale": 4.0, "retries": 0, "seconds": 312.3, "max_memory_gb": 17.17, "ce": 0.015966418548487127, "kl": 0.0040308857569471, "context": 0.017981861834414303}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 1.122620701789856, "loss_scale": 4.0, "retries": 0, "seconds": 313.6, "max_memory_gb": 17.17, "ce": 0.08294508513063192, "kl": 0.08684346848167479, "context": 0.12636681948788464}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.16318276524543762, "loss_scale": 4.0, "retries": 0, "seconds": 314.8, "max_memory_gb": 17.17, "ce": 0.009524301160126925, "kl": 0.005709864497475792, "context": 0.012379233521642163}

runs/fixed_internvl/context/Random8_8103_s13/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Random8_8103_s13/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.15.mlp.down_proj": 144,
+    "model.language_model.layers.15.mlp.gate_proj": 144,
+    "model.language_model.layers.15.mlp.up_proj": 144,
+    "model.language_model.layers.15.self_attn.k_proj": 144,
+    "model.language_model.layers.15.self_attn.o_proj": 144,
+    "model.language_model.layers.15.self_attn.q_proj": 144,
+    "model.language_model.layers.15.self_attn.v_proj": 144,
+    "model.language_model.layers.17.mlp.down_proj": 144,
+    "model.language_model.layers.17.mlp.gate_proj": 144,
+    "model.language_model.layers.17.mlp.up_proj": 144,
+    "model.language_model.layers.17.self_attn.k_proj": 144,
+    "model.language_model.layers.17.self_attn.o_proj": 144,
+    "model.language_model.layers.17.self_attn.q_proj": 144,
+    "model.language_model.layers.17.self_attn.v_proj": 144,
+    "model.language_model.layers.29.mlp.down_proj": 144,
+    "model.language_model.layers.29.mlp.gate_proj": 144,
+    "model.language_model.layers.29.mlp.up_proj": 144,
+    "model.language_model.layers.29.self_attn.k_proj": 144,
+    "model.language_model.layers.29.self_attn.o_proj": 144,
+    "model.language_model.layers.29.self_attn.q_proj": 144,
+    "model.language_model.layers.29.self_attn.v_proj": 144,
+    "model.language_model.layers.30.mlp.down_proj": 144,
+    "model.language_model.layers.30.mlp.gate_proj": 144,
+    "model.language_model.layers.30.mlp.up_proj": 144,
+    "model.language_model.layers.30.self_attn.k_proj": 144,
+    "model.language_model.layers.30.self_attn.o_proj": 144,
+    "model.language_model.layers.30.self_attn.q_proj": 144,
+    "model.language_model.layers.30.self_attn.v_proj": 144,
+    "model.language_model.layers.31.mlp.down_proj": 144,
+    "model.language_model.layers.31.mlp.gate_proj": 144,
+    "model.language_model.layers.31.mlp.up_proj": 144,
+    "model.language_model.layers.31.self_attn.k_proj": 144,
+    "model.language_model.layers.31.self_attn.o_proj": 144,
+    "model.language_model.layers.31.self_attn.q_proj": 144,
+    "model.language_model.layers.31.self_attn.v_proj": 144,
+    "model.language_model.layers.32.mlp.down_proj": 144,
+    "model.language_model.layers.32.mlp.gate_proj": 144,
+    "model.language_model.layers.32.mlp.up_proj": 144,
+    "model.language_model.layers.32.self_attn.k_proj": 144,
+    "model.language_model.layers.32.self_attn.o_proj": 144,
+    "model.language_model.layers.32.self_attn.q_proj": 144,
+    "model.language_model.layers.32.self_attn.v_proj": 144,
+    "model.language_model.layers.35.mlp.down_proj": 144,
+    "model.language_model.layers.35.mlp.gate_proj": 144,
+    "model.language_model.layers.35.mlp.up_proj": 144,
+    "model.language_model.layers.35.self_attn.k_proj": 144,
+    "model.language_model.layers.35.self_attn.o_proj": 144,
+    "model.language_model.layers.35.self_attn.q_proj": 144,
+    "model.language_model.layers.35.self_attn.v_proj": 144,
+    "model.language_model.layers.6.mlp.down_proj": 144,
+    "model.language_model.layers.6.mlp.gate_proj": 144,
+    "model.language_model.layers.6.mlp.up_proj": 144,
+    "model.language_model.layers.6.self_attn.k_proj": 144,
+    "model.language_model.layers.6.self_attn.o_proj": 144,
+    "model.language_model.layers.6.self_attn.q_proj": 144,
+    "model.language_model.layers.6.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.15.mlp.down_proj": 72,
+    "model.language_model.layers.15.mlp.gate_proj": 72,
+    "model.language_model.layers.15.mlp.up_proj": 72,
+    "model.language_model.layers.15.self_attn.k_proj": 72,
+    "model.language_model.layers.15.self_attn.o_proj": 72,
+    "model.language_model.layers.15.self_attn.q_proj": 72,
+    "model.language_model.layers.15.self_attn.v_proj": 72,
+    "model.language_model.layers.17.mlp.down_proj": 72,
+    "model.language_model.layers.17.mlp.gate_proj": 72,
+    "model.language_model.layers.17.mlp.up_proj": 72,
+    "model.language_model.layers.17.self_attn.k_proj": 72,
+    "model.language_model.layers.17.self_attn.o_proj": 72,
+    "model.language_model.layers.17.self_attn.q_proj": 72,
+    "model.language_model.layers.17.self_attn.v_proj": 72,
+    "model.language_model.layers.29.mlp.down_proj": 72,
+    "model.language_model.layers.29.mlp.gate_proj": 72,
+    "model.language_model.layers.29.mlp.up_proj": 72,
+    "model.language_model.layers.29.self_attn.k_proj": 72,
+    "model.language_model.layers.29.self_attn.o_proj": 72,
+    "model.language_model.layers.29.self_attn.q_proj": 72,
+    "model.language_model.layers.29.self_attn.v_proj": 72,
+    "model.language_model.layers.30.mlp.down_proj": 72,
+    "model.language_model.layers.30.mlp.gate_proj": 72,
+    "model.language_model.layers.30.mlp.up_proj": 72,
+    "model.language_model.layers.30.self_attn.k_proj": 72,
+    "model.language_model.layers.30.self_attn.o_proj": 72,
+    "model.language_model.layers.30.self_attn.q_proj": 72,
+    "model.language_model.layers.30.self_attn.v_proj": 72,
+    "model.language_model.layers.31.mlp.down_proj": 72,
+    "model.language_model.layers.31.mlp.gate_proj": 72,
+    "model.language_model.layers.31.mlp.up_proj": 72,
+    "model.language_model.layers.31.self_attn.k_proj": 72,
+    "model.language_model.layers.31.self_attn.o_proj": 72,
+    "model.language_model.layers.31.self_attn.q_proj": 72,
+    "model.language_model.layers.31.self_attn.v_proj": 72,
+    "model.language_model.layers.32.mlp.down_proj": 72,
+    "model.language_model.layers.32.mlp.gate_proj": 72,
+    "model.language_model.layers.32.mlp.up_proj": 72,
+    "model.language_model.layers.32.self_attn.k_proj": 72,
+    "model.language_model.layers.32.self_attn.o_proj": 72,
+    "model.language_model.layers.32.self_attn.q_proj": 72,
+    "model.language_model.layers.32.self_attn.v_proj": 72,
+    "model.language_model.layers.35.mlp.down_proj": 72,
+    "model.language_model.layers.35.mlp.gate_proj": 72,
+    "model.language_model.layers.35.mlp.up_proj": 72,
+    "model.language_model.layers.35.self_attn.k_proj": 72,
+    "model.language_model.layers.35.self_attn.o_proj": 72,
+    "model.language_model.layers.35.self_attn.q_proj": 72,
+    "model.language_model.layers.35.self_attn.v_proj": 72,
+    "model.language_model.layers.6.mlp.down_proj": 72,
+    "model.language_model.layers.6.mlp.gate_proj": 72,
+    "model.language_model.layers.6.mlp.up_proj": 72,
+    "model.language_model.layers.6.self_attn.k_proj": 72,
+    "model.language_model.layers.6.self_attn.o_proj": 72,
+    "model.language_model.layers.6.self_attn.q_proj": 72,
+    "model.language_model.layers.6.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.32.self_attn.o_proj",
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.30.mlp.down_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.35.self_attn.v_proj",
+    "model.language_model.layers.31.self_attn.q_proj",
+    "model.language_model.layers.32.mlp.down_proj",
+    "model.language_model.layers.17.self_attn.v_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.17.self_attn.o_proj",
+    "model.language_model.layers.31.mlp.up_proj",
+    "model.language_model.layers.29.self_attn.k_proj",
+    "model.language_model.layers.31.mlp.down_proj",
+    "model.language_model.layers.30.mlp.up_proj",
+    "model.language_model.layers.30.mlp.gate_proj",
+    "model.language_model.layers.6.self_attn.o_proj",
+    "model.language_model.layers.35.self_attn.o_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.30.self_attn.k_proj",
+    "model.language_model.layers.35.mlp.down_proj",
+    "model.language_model.layers.6.self_attn.k_proj",
+    "model.language_model.layers.6.self_attn.v_proj",
+    "model.language_model.layers.31.mlp.gate_proj",
+    "model.language_model.layers.35.self_attn.q_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.6.mlp.up_proj",
+    "model.language_model.layers.35.self_attn.k_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.30.self_attn.q_proj",
+    "model.language_model.layers.31.self_attn.k_proj",
+    "model.language_model.layers.31.self_attn.v_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.32.self_attn.k_proj",
+    "model.language_model.layers.35.mlp.up_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.35.mlp.gate_proj",
+    "model.language_model.layers.32.mlp.up_proj",
+    "model.language_model.layers.6.self_attn.q_proj",
+    "model.language_model.layers.6.mlp.gate_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.31.self_attn.o_proj",
+    "model.language_model.layers.30.self_attn.v_proj",
+    "model.language_model.layers.32.self_attn.q_proj",
+    "model.language_model.layers.29.self_attn.v_proj",
+    "model.language_model.layers.6.mlp.down_proj",
+    "model.language_model.layers.32.self_attn.v_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.30.self_attn.o_proj",
+    "model.language_model.layers.32.mlp.gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

runs/fixed_internvl/context/Random8_8103_s13/config.json

@@ -0,0 +1,43 @@
+{
+  "behavior": "context",
+  "seed": 13,
+  "model": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "backbone": "internvl",
+  "train": "/data/OPSD/assets/v3/context.jsonl",
+  "items": 1000,
+  "steps": 250,
+  "epochs": 1,
+  "steps_per_epoch": 250,
+  "world_size": 1,
+  "lr": 5e-05,
+  "scheduler": "cosine",
+  "warmup_steps": 16,
+  "final_fraction": 0.1,
+  "clip": 1.0,
+  "initial_loss_scale": 4.0,
+  "arm": {
+    "arm": "Random8_8103",
+    "layers": [
+      6,
+      15,
+      17,
+      29,
+      30,
+      31,
+      32,
+      35
+    ],
+    "ranks": {
+      "6": 72,
+      "15": 72,
+      "17": 72,
+      "29": 72,
+      "30": 72,
+      "31": 72,
+      "32": 72,
+      "35": 72
+    },
+    "parameters": 43646976,
+    "budget": 43646976
+  }
+}

runs/fixed_internvl/context/Random8_8103_s13/steps.jsonl

@@ -0,0 +1,250 @@
+{"step": 1, "steps": 250, "lr": 3.125e-06, "grad_norm": 6.465160846710205, "loss_scale": 4.0, "retries": 0, "seconds": 2.2, "max_memory_gb": 16.82, "ce": 0.5227766726748087, "kl": 0.08654405153356493, "context": 0.5660487064160407}
+{"step": 2, "steps": 250, "lr": 6.25e-06, "grad_norm": 6.8721208572387695, "loss_scale": 4.0, "retries": 0, "seconds": 3.5, "max_memory_gb": 17.16, "ce": 0.6328329627649509, "kl": 0.467102208429651, "context": 0.8663840865410748}
+{"step": 3, "steps": 250, "lr": 9.375000000000001e-06, "grad_norm": 17.297714233398438, "loss_scale": 4.0, "retries": 0, "seconds": 4.7, "max_memory_gb": 17.16, "ce": 0.9765701927244663, "kl": 0.07379033789038658, "context": 1.013465367257595}
+{"step": 4, "steps": 250, "lr": 1.25e-05, "grad_norm": 8.302188873291016, "loss_scale": 4.0, "retries": 0, "seconds": 6.0, "max_memory_gb": 17.16, "ce": 0.2761293037328869, "kl": 0.4529333900693473, "context": 0.5025960031343857}
+{"step": 5, "steps": 250, "lr": 1.5625e-05, "grad_norm": 0.2620261609554291, "loss_scale": 4.0, "retries": 0, "seconds": 7.2, "max_memory_gb": 17.16, "ce": 0.009469197801081464, "kl": 0.008649633462482598, "context": 0.013794014823361067}
+{"step": 6, "steps": 250, "lr": 1.8750000000000002e-05, "grad_norm": 2.8550832271575928, "loss_scale": 4.0, "retries": 0, "seconds": 8.4, "max_memory_gb": 17.16, "ce": 0.16553875603131019, "kl": 0.07969633290667844, "context": 0.20538692831178196}
+{"step": 7, "steps": 250, "lr": 2.1875e-05, "grad_norm": 4.9211931228637695, "loss_scale": 4.0, "retries": 0, "seconds": 9.7, "max_memory_gb": 17.16, "ce": 0.1348156527710671, "kl": 0.07494668444292074, "context": 0.17228899312976864}
+{"step": 8, "steps": 250, "lr": 2.5e-05, "grad_norm": 2.9434399604797363, "loss_scale": 4.0, "retries": 0, "seconds": 10.9, "max_memory_gb": 17.17, "ce": 0.07682205364108086, "kl": 0.21459126332774758, "context": 0.18411768414080143}
+{"step": 9, "steps": 250, "lr": 2.8125000000000003e-05, "grad_norm": 8.913880348205566, "loss_scale": 4.0, "retries": 0, "seconds": 12.1, "max_memory_gb": 17.17, "ce": 0.09104383178055286, "kl": 0.15602073553600349, "context": 0.16905420180410147}
+{"step": 10, "steps": 250, "lr": 3.125e-05, "grad_norm": 0.8388027548789978, "loss_scale": 4.0, "retries": 0, "seconds": 13.4, "max_memory_gb": 17.17, "ce": 0.0052043625473743305, "kl": 0.025068774120882154, "context": 0.01773875002982095}
+{"step": 11, "steps": 250, "lr": 3.4375e-05, "grad_norm": 0.18998107314109802, "loss_scale": 4.0, "retries": 0, "seconds": 14.7, "max_memory_gb": 17.17, "ce": 0.00544665037523373, "kl": 0.004411560697917594, "context": 0.007652430773305241}
+{"step": 12, "steps": 250, "lr": 3.7500000000000003e-05, "grad_norm": 0.31208881735801697, "loss_scale": 4.0, "retries": 0, "seconds": 15.8, "max_memory_gb": 17.17, "ce": 0.010190350123593817, "kl": 0.0021091130265631364, "context": 0.011244906869251281}
+{"step": 13, "steps": 250, "lr": 4.0625000000000005e-05, "grad_norm": 0.08002213388681412, "loss_scale": 4.0, "retries": 0, "seconds": 17.1, "max_memory_gb": 17.17, "ce": 0.001761429697580752, "kl": 0.0004548026863631094, "context": 0.0019888309980160557}
+{"step": 14, "steps": 250, "lr": 4.375e-05, "grad_norm": 3.273052215576172, "loss_scale": 4.0, "retries": 0, "seconds": 18.3, "max_memory_gb": 17.17, "ce": 0.055322830041404814, "kl": 0.7395675143925473, "context": 0.42510659410618246}
+{"step": 15, "steps": 250, "lr": 4.6875e-05, "grad_norm": 1.580932378768921, "loss_scale": 4.0, "retries": 0, "seconds": 19.6, "max_memory_gb": 17.17, "ce": 0.006393121020664694, "kl": 0.34225522781116524, "context": 0.17752073678889246}
+{"step": 16, "steps": 250, "lr": 5e-05, "grad_norm": 0.045696359127759933, "loss_scale": 4.0, "retries": 0, "seconds": 20.8, "max_memory_gb": 17.17, "ce": 0.0019337081503181253, "kl": 0.0007160640126357976, "context": 0.002291740194777958}
+{"step": 17, "steps": 250, "lr": 5e-05, "grad_norm": 2.539306879043579, "loss_scale": 4.0, "retries": 0, "seconds": 21.9, "max_memory_gb": 17.17, "ce": 0.023830845399061218, "kl": 0.8175140931271017, "context": 0.432587895920733}
+{"step": 18, "steps": 250, "lr": 4.999797225177623e-05, "grad_norm": 1.0841176509857178, "loss_scale": 4.0, "retries": 0, "seconds": 23.2, "max_memory_gb": 17.17, "ce": 0.0073617307643871754, "kl": 0.249431474301673, "context": 0.13207746544503607}
+{"step": 19, "steps": 250, "lr": 4.999188937259492e-05, "grad_norm": 0.5065702199935913, "loss_scale": 4.0, "retries": 0, "seconds": 24.4, "max_memory_gb": 17.17, "ce": 0.006312758177500655, "kl": 0.05587089241180365, "context": 0.034248204736286425}
+{"step": 20, "steps": 250, "lr": 4.998175245886032e-05, "grad_norm": 0.1615883708000183, "loss_scale": 4.0, "retries": 0, "seconds": 25.6, "max_memory_gb": 17.17, "ce": 0.0009821731973715941, "kl": 0.00926995433292177, "context": 0.005617150221951306}
+{"step": 21, "steps": 250, "lr": 4.996756333769319e-05, "grad_norm": 0.03923941031098366, "loss_scale": 4.0, "retries": 0, "seconds": 27.1, "max_memory_gb": 17.17, "ce": 0.000455504019100772, "kl": 0.001625219825655222, "context": 0.0012681139214691939}
+{"step": 22, "steps": 250, "lr": 4.994932456660157e-05, "grad_norm": 2.513134479522705, "loss_scale": 4.0, "retries": 0, "seconds": 28.4, "max_memory_gb": 17.17, "ce": 0.08838471169292461, "kl": 0.06697899845903521, "context": 0.12187421022099443}
+{"step": 23, "steps": 250, "lr": 4.992703943301973e-05, "grad_norm": 3.5475375652313232, "loss_scale": 4.0, "retries": 0, "seconds": 29.7, "max_memory_gb": 17.17, "ce": 0.00014179473555486766, "kl": 0.2269920165123267, "context": 0.11363780322835737}
+{"step": 24, "steps": 250, "lr": 4.9900711953715675e-05, "grad_norm": 0.7184044718742371, "loss_scale": 4.0, "retries": 0, "seconds": 31.0, "max_memory_gb": 17.17, "ce": 0.008256012602942064, "kl": 0.031238662526448024, "context": 0.023875343998952303}
+{"step": 25, "steps": 250, "lr": 4.987034687406713e-05, "grad_norm": 3.419097661972046, "loss_scale": 4.0, "retries": 0, "seconds": 32.4, "max_memory_gb": 17.17, "ce": 0.16127902732023358, "kl": 0.027191940927878022, "context": 0.17487500159768388}
+{"step": 26, "steps": 250, "lr": 4.9835949667206215e-05, "grad_norm": 1.6097354888916016, "loss_scale": 4.0, "retries": 0, "seconds": 33.6, "max_memory_gb": 17.17, "ce": 0.044789246873733646, "kl": 0.5252820341188453, "context": 0.3074302565109974}
+{"step": 27, "steps": 250, "lr": 4.9797526533032945e-05, "grad_norm": 1.648429036140442, "loss_scale": 4.0, "retries": 0, "seconds": 34.9, "max_memory_gb": 17.17, "ce": 0.05252725198806729, "kl": 0.29005332760198144, "context": 0.19755391258149757}
+{"step": 28, "steps": 250, "lr": 4.9755084397097716e-05, "grad_norm": 2.840235710144043, "loss_scale": 4.0, "retries": 0, "seconds": 36.1, "max_memory_gb": 17.17, "ce": 0.12083835243242902, "kl": 0.25954724522307515, "context": 0.2506119729951024}
+{"step": 29, "steps": 250, "lr": 4.970863090935304e-05, "grad_norm": 5.143810749053955, "loss_scale": 4.0, "retries": 0, "seconds": 37.4, "max_memory_gb": 17.17, "ce": 0.5035758567973971, "kl": 0.13394373751361854, "context": 0.5705477211158723}
+{"step": 30, "steps": 250, "lr": 4.965817444277468e-05, "grad_norm": 0.5022260546684265, "loss_scale": 4.0, "retries": 0, "seconds": 38.7, "max_memory_gb": 17.17, "ce": 0.003810458330917754, "kl": 0.019191888135537738, "context": 0.01340640213493316}
+{"step": 31, "steps": 250, "lr": 4.9603724091852456e-05, "grad_norm": 8.454697608947754, "loss_scale": 4.0, "retries": 0, "seconds": 39.9, "max_memory_gb": 17.17, "ce": 0.6624914036510745, "kl": 0.08914211246883497, "context": 0.7070624514017254}
+{"step": 32, "steps": 250, "lr": 4.9545289670951025e-05, "grad_norm": 1.665898323059082, "loss_scale": 4.0, "retries": 0, "seconds": 41.2, "max_memory_gb": 17.17, "ce": 0.042580985464155674, "kl": 0.06627023273904342, "context": 0.0757161017972976}
+{"step": 33, "steps": 250, "lr": 4.948288171254089e-05, "grad_norm": 4.009949684143066, "loss_scale": 4.0, "retries": 0, "seconds": 42.4, "max_memory_gb": 17.17, "ce": 0.3326260303147137, "kl": 0.06434644025284797, "context": 0.3647992566693574}
+{"step": 34, "steps": 250, "lr": 4.941651146529998e-05, "grad_norm": 1.3278989791870117, "loss_scale": 4.0, "retries": 0, "seconds": 43.7, "max_memory_gb": 17.17, "ce": 0.006937006110092625, "kl": 0.7715615264023654, "context": 0.3927177799050696}
+{"step": 35, "steps": 250, "lr": 4.9346190892086174e-05, "grad_norm": 0.051304012537002563, "loss_scale": 4.0, "retries": 0, "seconds": 45.0, "max_memory_gb": 17.17, "ce": 0.002078299963613972, "kl": 0.00040239595546154305, "context": 0.002279497879499104}
+{"step": 36, "steps": 250, "lr": 4.927193266778101e-05, "grad_norm": 3.082399368286133, "loss_scale": 4.0, "retries": 0, "seconds": 46.2, "max_memory_gb": 17.17, "ce": 0.23020930515485816, "kl": 0.030998405170976184, "context": 0.24570850495365448}
+{"step": 37, "steps": 250, "lr": 4.919375017700515e-05, "grad_norm": 2.5118649005889893, "loss_scale": 4.0, "retries": 0, "seconds": 47.5, "max_memory_gb": 17.17, "ce": 0.11056598118011607, "kl": 0.4415202116360888, "context": 0.33132609364110976}
+{"step": 38, "steps": 250, "lr": 4.911165751170588e-05, "grad_norm": 1.6803083419799805, "loss_scale": 4.0, "retries": 0, "seconds": 49.0, "max_memory_gb": 17.17, "ce": 0.08068710926454514, "kl": 0.41873526634299196, "context": 0.2900547455647029}
+{"step": 39, "steps": 250, "lr": 4.902566946861708e-05, "grad_norm": 1.482689619064331, "loss_scale": 4.0, "retries": 0, "seconds": 50.5, "max_memory_gb": 17.17, "ce": 0.07335664924175944, "kl": 0.009945375040842919, "context": 0.078329338633921}
+{"step": 40, "steps": 250, "lr": 4.89358015465922e-05, "grad_norm": 0.40771833062171936, "loss_scale": 4.0, "retries": 0, "seconds": 52.0, "max_memory_gb": 17.17, "ce": 0.01663652842398733, "kl": 0.02212497970322147, "context": 0.0276990185957402}
+{"step": 41, "steps": 250, "lr": 4.884206994381078e-05, "grad_norm": 3.729485273361206, "loss_scale": 4.0, "retries": 0, "seconds": 53.2, "max_memory_gb": 17.17, "ce": 0.5345431219320744, "kl": 0.07518196012824774, "context": 0.5721341110765934}
+{"step": 42, "steps": 250, "lr": 4.874449155485866e-05, "grad_norm": 0.13966794312000275, "loss_scale": 4.0, "retries": 0, "seconds": 54.4, "max_memory_gb": 17.17, "ce": 0.008169030159479007, "kl": 0.0010755461116787046, "context": 0.008706803142558783}
+{"step": 43, "steps": 250, "lr": 4.864308396768294e-05, "grad_norm": 0.15157559514045715, "loss_scale": 4.0, "retries": 0, "seconds": 55.7, "max_memory_gb": 17.17, "ce": 0.006355586228892207, "kl": 0.0010869620564335492, "context": 0.00689906720072031}
+{"step": 44, "steps": 250, "lr": 4.853786546042184e-05, "grad_norm": 14.681798934936523, "loss_scale": 4.0, "retries": 0, "seconds": 56.8, "max_memory_gb": 17.17, "ce": 0.10010069340933114, "kl": 0.757932389547932, "context": 0.4790668752975762}
+{"step": 45, "steps": 250, "lr": 4.842885499811012e-05, "grad_norm": 4.790704727172852, "loss_scale": 4.0, "retries": 0, "seconds": 58.1, "max_memory_gb": 17.17, "ce": 0.4705290269339457, "kl": 0.18160582688869908, "context": 0.5613319560652599}
+{"step": 46, "steps": 250, "lr": 4.831607222926078e-05, "grad_norm": 1.1722170114517212, "loss_scale": 4.0, "retries": 0, "seconds": 59.8, "max_memory_gb": 17.17, "ce": 0.05899990981561132, "kl": 0.11790167522849515, "context": 0.11795074463589117}
+{"step": 47, "steps": 250, "lr": 4.8199537482323545e-05, "grad_norm": 1.1121649742126465, "loss_scale": 4.0, "retries": 0, "seconds": 61.1, "max_memory_gb": 17.17, "ce": 0.030378989467862993, "kl": 0.15196319955430226, "context": 0.10636058659292758}
+{"step": 48, "steps": 250, "lr": 4.807927176202073e-05, "grad_norm": 0.09337456524372101, "loss_scale": 4.0, "retries": 0, "seconds": 62.5, "max_memory_gb": 17.17, "ce": 0.0036284634843468666, "kl": 0.002404866283541196, "context": 0.004830896621569991}
+{"step": 49, "steps": 250, "lr": 4.795529674556129e-05, "grad_norm": 0.35129615664482117, "loss_scale": 4.0, "retries": 0, "seconds": 63.6, "max_memory_gb": 17.17, "ce": 0.012313948056544177, "kl": 0.03149830770416884, "context": 0.028063101548468694}
+{"step": 50, "steps": 250, "lr": 4.782763477873361e-05, "grad_norm": 0.9769709706306458, "loss_scale": 4.0, "retries": 0, "seconds": 64.8, "max_memory_gb": 17.17, "ce": 0.06817365807364695, "kl": 0.013171800892450847, "context": 0.0747595599386841}
+{"step": 51, "steps": 250, "lr": 4.769630887187782e-05, "grad_norm": 74.21534729003906, "loss_scale": 4.0, "retries": 0, "seconds": 66.1, "max_memory_gb": 17.17, "ce": 0.10551891599607188, "kl": 0.027288843535643537, "context": 0.11916333946282975}
+{"step": 52, "steps": 250, "lr": 4.756134269573832e-05, "grad_norm": 0.5661571621894836, "loss_scale": 4.0, "retries": 0, "seconds": 67.3, "max_memory_gb": 17.17, "ce": 0.018309711711481214, "kl": 0.12823828172986396, "context": 0.0824288516305387}
+{"step": 53, "steps": 250, "lr": 4.7422760577197226e-05, "grad_norm": 2.187244415283203, "loss_scale": 4.0, "retries": 0, "seconds": 68.5, "max_memory_gb": 17.17, "ce": 0.0342591597000137, "kl": 0.20916530676186085, "context": 0.1388418124988675}
+{"step": 54, "steps": 250, "lr": 4.728058749488963e-05, "grad_norm": 3.9263808727264404, "loss_scale": 4.0, "retries": 0, "seconds": 69.9, "max_memory_gb": 17.17, "ce": 0.12241283472394571, "kl": 0.0735312798205996, "context": 0.1591784667398315}
+{"step": 55, "steps": 250, "lr": 4.713484907470133e-05, "grad_norm": 0.2849608361721039, "loss_scale": 4.0, "retries": 0, "seconds": 71.1, "max_memory_gb": 17.17, "ce": 0.017327110865153372, "kl": 0.008384492492041318, "context": 0.02151935617439449}
+{"step": 56, "steps": 250, "lr": 4.6985571585149876e-05, "grad_norm": 0.6317854523658752, "loss_scale": 4.0, "retries": 0, "seconds": 72.3, "max_memory_gb": 17.17, "ce": 0.02093345602042973, "kl": 0.08102821602369659, "context": 0.06144756404682994}
+{"step": 57, "steps": 250, "lr": 4.6832781932649884e-05, "grad_norm": 0.21218766272068024, "loss_scale": 4.0, "retries": 0, "seconds": 73.7, "max_memory_gb": 17.17, "ce": 0.00851527982740663, "kl": 0.012413390664733015, "context": 0.014721974730491638}
+{"step": 58, "steps": 250, "lr": 4.667650765666325e-05, "grad_norm": 0.0355093739926815, "loss_scale": 4.0, "retries": 0, "seconds": 75.0, "max_memory_gb": 17.17, "ce": 0.0015325329441111535, "kl": 0.00022188817388268944, "context": 0.0016434770222986117}
+{"step": 59, "steps": 250, "lr": 4.651677692473538e-05, "grad_norm": 1.6486616134643555, "loss_scale": 4.0, "retries": 0, "seconds": 76.2, "max_memory_gb": 17.17, "ce": 0.12612415527109988, "kl": 0.05919879913562909, "context": 0.15572355344193056}
+{"step": 60, "steps": 250, "lr": 4.635361852741808e-05, "grad_norm": 1.498339295387268, "loss_scale": 4.0, "retries": 0, "seconds": 77.5, "max_memory_gb": 17.17, "ce": 0.10672877746401355, "kl": 0.08398757914255839, "context": 0.14872255956288427}
+{"step": 61, "steps": 250, "lr": 4.6187061873080264e-05, "grad_norm": 1.2710670232772827, "loss_scale": 4.0, "retries": 0, "seconds": 78.8, "max_memory_gb": 17.17, "ce": 0.06348029657965526, "kl": 0.49934852110163774, "context": 0.31315454782452434}
+{"step": 62, "steps": 250, "lr": 4.601713698260727e-05, "grad_norm": 0.621258556842804, "loss_scale": 4.0, "retries": 0, "seconds": 80.1, "max_memory_gb": 17.17, "ce": 0.03622353609534912, "kl": 0.008468346066365484, "context": 0.04045770870288834}
+{"step": 63, "steps": 250, "lr": 4.5843874483989746e-05, "grad_norm": 1.1258339881896973, "loss_scale": 4.0, "retries": 0, "seconds": 81.4, "max_memory_gb": 17.17, "ce": 0.05732681122026406, "kl": 0.07579146485659294, "context": 0.09522254392504692}
+{"step": 64, "steps": 250, "lr": 4.566730560680316e-05, "grad_norm": 0.22060555219650269, "loss_scale": 4.0, "retries": 0, "seconds": 82.6, "max_memory_gb": 17.17, "ce": 0.011884224513778463, "kl": 0.0051434079359751195, "context": 0.014455928816460073}
+{"step": 65, "steps": 250, "lr": 4.548746217657878e-05, "grad_norm": 0.9983551502227783, "loss_scale": 4.0, "retries": 0, "seconds": 83.9, "max_memory_gb": 17.17, "ce": 0.12719666259363294, "kl": 0.10826433962211013, "context": 0.18132883217185736}
+{"step": 66, "steps": 250, "lr": 4.5304376609067396e-05, "grad_norm": 2.3899872303009033, "loss_scale": 4.0, "retries": 0, "seconds": 85.1, "max_memory_gb": 17.17, "ce": 0.1780473814542347, "kl": 0.08421764422814704, "context": 0.22015619611738657}
+{"step": 67, "steps": 250, "lr": 4.5118081904396504e-05, "grad_norm": 0.08135674893856049, "loss_scale": 4.0, "retries": 0, "seconds": 86.4, "max_memory_gb": 17.17, "ce": 0.004606695641996339, "kl": 0.005572141058109992, "context": 0.0073927660851040855}
+{"step": 68, "steps": 250, "lr": 4.4928611641122225e-05, "grad_norm": 0.3119796812534332, "loss_scale": 4.0, "retries": 0, "seconds": 87.7, "max_memory_gb": 17.17, "ce": 0.009233217519067694, "kl": 0.012940153827003087, "context": 0.01570329489914002}
+{"step": 69, "steps": 250, "lr": 4.473599997017701e-05, "grad_norm": 4.257806777954102, "loss_scale": 4.0, "retries": 0, "seconds": 88.9, "max_memory_gb": 17.17, "ce": 0.6173348785087001, "kl": 0.009920491851517, "context": 0.6222950988449156}
+{"step": 70, "steps": 250, "lr": 4.4540281608714053e-05, "grad_norm": 0.7985284328460693, "loss_scale": 4.0, "retries": 0, "seconds": 90.3, "max_memory_gb": 17.17, "ce": 0.010411748779006302, "kl": 0.0010903028896791511, "context": 0.010956900689052418}
+{"step": 71, "steps": 250, "lr": 4.434149183384977e-05, "grad_norm": 0.19179047644138336, "loss_scale": 4.0, "retries": 0, "seconds": 91.5, "max_memory_gb": 17.17, "ce": 0.007355606729106512, "kl": 0.006464625534135848, "context": 0.010587919499812415}
+{"step": 72, "steps": 250, "lr": 4.413966647630534e-05, "grad_norm": 1.0332285165786743, "loss_scale": 4.0, "retries": 0, "seconds": 92.7, "max_memory_gb": 17.17, "ce": 0.010687714966479689, "kl": 0.42517321603372693, "context": 0.22327432298334315}
+{"step": 73, "steps": 250, "lr": 4.3934841913948323e-05, "grad_norm": 2.3992350101470947, "loss_scale": 4.0, "retries": 0, "seconds": 93.9, "max_memory_gb": 17.17, "ce": 0.16176170289327274, "kl": 0.03979376959432557, "context": 0.18165858210340957}
+{"step": 74, "steps": 250, "lr": 4.372705506523583e-05, "grad_norm": 1.4523428678512573, "loss_scale": 4.0, "retries": 0, "seconds": 95.1, "max_memory_gb": 17.17, "ce": 0.11096797566278838, "kl": 0.06444983272012905, "context": 0.14319289194827434}
+{"step": 75, "steps": 250, "lr": 4.351634338256017e-05, "grad_norm": 2.4224002361297607, "loss_scale": 4.0, "retries": 0, "seconds": 96.2, "max_memory_gb": 17.17, "ce": 0.5205158495227806, "kl": 0.8201859639957547, "context": 0.930608844384551}
+{"step": 76, "steps": 250, "lr": 4.330274484549825e-05, "grad_norm": 2.4840333461761475, "loss_scale": 4.0, "retries": 0, "seconds": 97.3, "max_memory_gb": 17.17, "ce": 0.015270567608240526, "kl": 1.3716207772959024, "context": 0.7010809452040121}
+{"step": 77, "steps": 250, "lr": 4.308629795396599e-05, "grad_norm": 0.31776005029678345, "loss_scale": 4.0, "retries": 0, "seconds": 98.3, "max_memory_gb": 17.17, "ce": 0.013428752834443003, "kl": 0.013044349558185786, "context": 0.019950927584432065}
+{"step": 78, "steps": 250, "lr": 4.286704172127894e-05, "grad_norm": 0.6812352538108826, "loss_scale": 4.0, "retries": 0, "seconds": 99.3, "max_memory_gb": 17.17, "ce": 0.0020510139293037355, "kl": 0.09487335032827104, "context": 0.04948768956819549}
+{"step": 79, "steps": 250, "lr": 4.264501566712033e-05, "grad_norm": 0.6499646902084351, "loss_scale": 4.0, "retries": 0, "seconds": 100.4, "max_memory_gb": 17.17, "ce": 0.014367087456776062, "kl": 0.019489652468109853, "context": 0.024111913749948144}
+{"step": 80, "steps": 250, "lr": 4.2420259810417894e-05, "grad_norm": 0.5666939616203308, "loss_scale": 4.0, "retries": 0, "seconds": 101.6, "max_memory_gb": 17.17, "ce": 0.0029322368354769424, "kl": 0.06218639056169195, "context": 0.03402543326228624}
+{"step": 81, "steps": 250, "lr": 4.219281466213066e-05, "grad_norm": 1.0708647966384888, "loss_scale": 4.0, "retries": 0, "seconds": 103.0, "max_memory_gb": 17.17, "ce": 0.0857317985210102, "kl": 0.03845423203529208, "context": 0.10495891547179781}
+{"step": 82, "steps": 250, "lr": 4.1962721217947135e-05, "grad_norm": 0.15479056537151337, "loss_scale": 4.0, "retries": 0, "seconds": 104.3, "max_memory_gb": 17.17, "ce": 0.009754379920195788, "kl": 0.002076152231893502, "context": 0.010792455985210836}
+{"step": 83, "steps": 250, "lr": 4.173002095089599e-05, "grad_norm": 2.042902708053589, "loss_scale": 4.0, "retries": 0, "seconds": 105.5, "max_memory_gb": 17.17, "ce": 0.10524639867071528, "kl": 0.5896415357565274, "context": 0.40006717515643686}
+{"step": 84, "steps": 250, "lr": 4.149475580387084e-05, "grad_norm": 2.0793795585632324, "loss_scale": 4.0, "retries": 0, "seconds": 106.9, "max_memory_gb": 17.17, "ce": 0.12255006056147977, "kl": 0.041958124420489185, "context": 0.1435291210327705}
+{"step": 85, "steps": 250, "lr": 4.1256968182070275e-05, "grad_norm": 0.2096010148525238, "loss_scale": 4.0, "retries": 0, "seconds": 108.1, "max_memory_gb": 17.17, "ce": 0.004421678717335453, "kl": 0.002028794778198062, "context": 0.0054360762260330375}
+{"step": 86, "steps": 250, "lr": 4.101670094535453e-05, "grad_norm": 2.0062637329101562, "loss_scale": 4.0, "retries": 0, "seconds": 109.3, "max_memory_gb": 17.17, "ce": 0.02947636489989236, "kl": 0.2565120716462843, "context": 0.15773240476846695}
+{"step": 87, "steps": 250, "lr": 4.077399740052027e-05, "grad_norm": 2.2301764488220215, "loss_scale": 4.0, "retries": 0, "seconds": 110.6, "max_memory_gb": 17.17, "ce": 0.006767240818589926, "kl": 0.581004099920392, "context": 0.2972692947369069}
+{"step": 88, "steps": 250, "lr": 4.0528901293494775e-05, "grad_norm": 0.2961882948875427, "loss_scale": 4.0, "retries": 0, "seconds": 111.8, "max_memory_gb": 17.17, "ce": 0.014150999526464147, "kl": 0.002893848928749776, "context": 0.0155979241571913}
+{"step": 89, "steps": 250, "lr": 4.0281456801451004e-05, "grad_norm": 0.3467867374420166, "loss_scale": 4.0, "retries": 0, "seconds": 112.9, "max_memory_gb": 17.17, "ce": 0.021710746186727192, "kl": 0.012096506397938356, "context": 0.027758998388890177}
+{"step": 90, "steps": 250, "lr": 4.0031708524844884e-05, "grad_norm": 0.1938440352678299, "loss_scale": 4.0, "retries": 0, "seconds": 114.2, "max_memory_gb": 17.17, "ce": 0.012165072810603306, "kl": 0.004438412535819225, "context": 0.014384279551450163}
+{"step": 91, "steps": 250, "lr": 3.977970147937635e-05, "grad_norm": 4.536449909210205, "loss_scale": 4.0, "retries": 0, "seconds": 115.5, "max_memory_gb": 17.17, "ce": 0.4181772614538204, "kl": 0.250782856324804, "context": 0.5435686816927046}
+{"step": 92, "steps": 250, "lr": 3.952548108787553e-05, "grad_norm": 5.7627973556518555, "loss_scale": 4.0, "retries": 0, "seconds": 116.7, "max_memory_gb": 17.17, "ce": 0.17083302366518183, "kl": 0.2658160071287057, "context": 0.303741013056424}
+{"step": 93, "steps": 250, "lr": 3.9269093172115496e-05, "grad_norm": 4.440215110778809, "loss_scale": 4.0, "retries": 0, "seconds": 118.0, "max_memory_gb": 17.17, "ce": 0.06612139526987448, "kl": 0.3423095424150233, "context": 0.23727617121767253}
+{"step": 94, "steps": 250, "lr": 3.901058394455323e-05, "grad_norm": 3.5516979694366455, "loss_scale": 4.0, "retries": 0, "seconds": 119.2, "max_memory_gb": 17.17, "ce": 0.08336079306900501, "kl": 0.03262980740692001, "context": 0.09967569299624301}
+{"step": 95, "steps": 250, "lr": 3.875e-05, "grad_norm": 4.520626068115234, "loss_scale": 4.0, "retries": 0, "seconds": 120.4, "max_memory_gb": 17.17, "ce": 0.4869800776723423, "kl": 0.017350756912492216, "context": 0.49565545696532354}
+{"step": 96, "steps": 250, "lr": 3.8487388307223005e-05, "grad_norm": 3.0473830699920654, "loss_scale": 4.0, "retries": 0, "seconds": 121.6, "max_memory_gb": 17.17, "ce": 0.31407155966735445, "kl": 0.0051567580558185, "context": 0.3166499499056954}
+{"step": 97, "steps": 250, "lr": 3.822279620047943e-05, "grad_norm": 0.3191252648830414, "loss_scale": 4.0, "retries": 0, "seconds": 122.8, "max_memory_gb": 17.17, "ce": 0.012937857885845006, "kl": 0.010551540413871408, "context": 0.018213627859950066}
+{"step": 98, "steps": 250, "lr": 3.795627137098479e-05, "grad_norm": 0.5707310438156128, "loss_scale": 4.0, "retries": 0, "seconds": 124.0, "max_memory_gb": 17.17, "ce": 0.0020321187694207765, "kl": 0.09583677019327297, "context": 0.04995050259458367}
+{"step": 99, "steps": 250, "lr": 3.7687861858316844e-05, "grad_norm": 0.17213821411132812, "loss_scale": 4.0, "retries": 0, "seconds": 125.2, "max_memory_gb": 17.17, "ce": 0.014304180640465347, "kl": 0.005821603308504564, "context": 0.017214982755831443}
+{"step": 100, "steps": 250, "lr": 3.741761604175668e-05, "grad_norm": 1.1026790142059326, "loss_scale": 4.0, "retries": 0, "seconds": 126.4, "max_memory_gb": 17.17, "ce": 0.02962814152124338, "kl": 0.36592462840053486, "context": 0.21259045042097569}
+{"step": 101, "steps": 250, "lr": 3.714558263156872e-05, "grad_norm": 0.3627065420150757, "loss_scale": 4.0, "retries": 0, "seconds": 127.6, "max_memory_gb": 17.17, "ce": 0.0038772512052673846, "kl": 0.03277556585089769, "context": 0.02026503425440751}
+{"step": 102, "steps": 250, "lr": 3.6871810660220904e-05, "grad_norm": 0.4234647750854492, "loss_scale": 4.0, "retries": 0, "seconds": 128.9, "max_memory_gb": 17.17, "ce": 0.03002671169815585, "kl": 0.009487687872024253, "context": 0.03477055608527735}
+{"step": 103, "steps": 250, "lr": 3.659634947354686e-05, "grad_norm": 0.23425062000751495, "loss_scale": 4.0, "retries": 0, "seconds": 130.2, "max_memory_gb": 17.17, "ce": 0.012345000373898074, "kl": 0.0069993725228414405, "context": 0.015844686233322136}
+{"step": 104, "steps": 250, "lr": 3.631924872185169e-05, "grad_norm": 0.21572840213775635, "loss_scale": 4.0, "retries": 0, "seconds": 131.4, "max_memory_gb": 17.17, "ce": 0.010961134365061298, "kl": 0.0036984952421335038, "context": 0.012810381973395124}
+{"step": 105, "steps": 250, "lr": 3.6040558350962674e-05, "grad_norm": 1.3246744871139526, "loss_scale": 4.0, "retries": 0, "seconds": 132.6, "max_memory_gb": 17.17, "ce": 0.015479060122743249, "kl": 0.29283414263045415, "context": 0.16189612829475664}
+{"step": 106, "steps": 250, "lr": 3.576032859322687e-05, "grad_norm": 0.03477517515420914, "loss_scale": 4.0, "retries": 0, "seconds": 133.9, "max_memory_gb": 17.17, "ce": 0.002527352888137102, "kl": 0.0004699703131336719, "context": 0.002762338030152023}
+{"step": 107, "steps": 250, "lr": 3.5478609958457056e-05, "grad_norm": 0.8348162174224854, "loss_scale": 4.0, "retries": 0, "seconds": 135.1, "max_memory_gb": 17.17, "ce": 0.04265331268834416, "kl": 0.15511557494340877, "context": 0.12021109968191013}
+{"step": 108, "steps": 250, "lr": 3.519545322482755e-05, "grad_norm": 3.3184962272644043, "loss_scale": 4.0, "retries": 0, "seconds": 136.3, "max_memory_gb": 17.17, "ce": 0.28525386755063664, "kl": 0.00751393456664573, "context": 0.2890108442516066}
+{"step": 109, "steps": 250, "lr": 3.491090942972179e-05, "grad_norm": 4.109201431274414, "loss_scale": 4.0, "retries": 0, "seconds": 137.6, "max_memory_gb": 17.17, "ce": 0.2756825778342318, "kl": 0.17860345914959908, "context": 0.36498431907966733}
+{"step": 110, "steps": 250, "lr": 3.4625029860533135e-05, "grad_norm": 0.9904748797416687, "loss_scale": 4.0, "retries": 0, "seconds": 138.8, "max_memory_gb": 17.17, "ce": 0.059571942780166864, "kl": 0.005721712368540466, "context": 0.06243279995396733}
+{"step": 111, "steps": 250, "lr": 3.433786604542056e-05, "grad_norm": 0.020677600055933, "loss_scale": 4.0, "retries": 0, "seconds": 139.9, "max_memory_gb": 17.17, "ce": 0.0013636660150950775, "kl": 0.0007417824199364986, "context": 0.001734557212330401}
+{"step": 112, "steps": 250, "lr": 3.404946974402104e-05, "grad_norm": 0.20416633784770966, "loss_scale": 4.0, "retries": 0, "seconds": 141.3, "max_memory_gb": 17.17, "ce": 0.010604396578855813, "kl": 0.002588562269011163, "context": 0.011898678028956056}
+{"step": 113, "steps": 250, "lr": 3.375989293812019e-05, "grad_norm": 0.9796725511550903, "loss_scale": 4.0, "retries": 0, "seconds": 142.5, "max_memory_gb": 17.17, "ce": 0.007735329156275839, "kl": 0.07384291919879615, "context": 0.04465678858105093}
+{"step": 114, "steps": 250, "lr": 3.3469187822282766e-05, "grad_norm": 2.953411102294922, "loss_scale": 4.0, "retries": 0, "seconds": 143.7, "max_memory_gb": 17.17, "ce": 0.015692252942244522, "kl": 0.3719205887118733, "context": 0.20165254939638544}
+{"step": 115, "steps": 250, "lr": 3.317740679444499e-05, "grad_norm": 1.6571109294891357, "loss_scale": 4.0, "retries": 0, "seconds": 144.9, "max_memory_gb": 17.17, "ce": 0.04808818828314543, "kl": 0.09764866715704557, "context": 0.09691251849289984}
+{"step": 116, "steps": 250, "lr": 3.2884602446470056e-05, "grad_norm": 1.4761093854904175, "loss_scale": 4.0, "retries": 0, "seconds": 146.1, "max_memory_gb": 17.17, "ce": 0.1166089587204624, "kl": 0.2789217514837219, "context": 0.25606981955934316}
+{"step": 117, "steps": 250, "lr": 3.259082755466876e-05, "grad_norm": 0.07463154196739197, "loss_scale": 4.0, "retries": 0, "seconds": 147.4, "max_memory_gb": 17.17, "ce": 0.0040285742143169045, "kl": 0.0023083648702595383, "context": 0.005182756649446674}
+{"step": 118, "steps": 250, "lr": 3.229613507028686e-05, "grad_norm": 1.7122808694839478, "loss_scale": 4.0, "retries": 0, "seconds": 148.6, "max_memory_gb": 17.17, "ce": 0.24481208104407415, "kl": 0.004126423661546141, "context": 0.24687529749644455}
+{"step": 119, "steps": 250, "lr": 3.2000578109961004e-05, "grad_norm": 0.15923014283180237, "loss_scale": 4.0, "retries": 0, "seconds": 149.8, "max_memory_gb": 17.17, "ce": 0.004962265738868155, "kl": 0.004819026148084049, "context": 0.0073717787163332105}
+{"step": 120, "steps": 250, "lr": 3.1704209946144616e-05, "grad_norm": 4.179967880249023, "loss_scale": 4.0, "retries": 0, "seconds": 151.1, "max_memory_gb": 17.17, "ce": 0.20861384295858443, "kl": 0.2377310564625077, "context": 0.32747936935629696}
+{"step": 121, "steps": 250, "lr": 3.140708399750594e-05, "grad_norm": 1.0179139375686646, "loss_scale": 4.0, "retries": 0, "seconds": 152.3, "max_memory_gb": 17.17, "ce": 0.06286589301453205, "kl": 0.05002406234416412, "context": 0.08787792603834532}
+{"step": 122, "steps": 250, "lr": 3.1109253819299606e-05, "grad_norm": 1.051767349243164, "loss_scale": 4.0, "retries": 0, "seconds": 153.5, "max_memory_gb": 17.17, "ce": 0.12127101469377521, "kl": 0.015968013864039676, "context": 0.12925501976860687}
+{"step": 123, "steps": 250, "lr": 3.081077309371361e-05, "grad_norm": 1.227315068244934, "loss_scale": 4.0, "retries": 0, "seconds": 155.0, "max_memory_gb": 17.17, "ce": 0.06595432001631707, "kl": 0.03891804697923362, "context": 0.08541334373876452}
+{"step": 124, "steps": 250, "lr": 3.051169562019339e-05, "grad_norm": 3.839812994003296, "loss_scale": 4.0, "retries": 0, "seconds": 156.2, "max_memory_gb": 17.17, "ce": 0.6231575032434193, "kl": 0.4676054702372312, "context": 0.8569602541974746}
+{"step": 125, "steps": 250, "lr": 3.021207530574477e-05, "grad_norm": 1.1979789733886719, "loss_scale": 4.0, "retries": 0, "seconds": 157.5, "max_memory_gb": 17.17, "ce": 0.13909479341236874, "kl": 0.07915660874641617, "context": 0.1786731014144607}
+{"step": 126, "steps": 250, "lr": 2.991196615521758e-05, "grad_norm": 3.990955114364624, "loss_scale": 4.0, "retries": 0, "seconds": 158.9, "max_memory_gb": 17.17, "ce": 0.699930964037776, "kl": 0.07656939746811986, "context": 0.738215658813715}
+{"step": 127, "steps": 250, "lr": 2.9611422261571536e-05, "grad_norm": 0.05027870461344719, "loss_scale": 4.0, "retries": 0, "seconds": 160.2, "max_memory_gb": 17.17, "ce": 0.0030309689027490094, "kl": 0.00032321803519153036, "context": 0.0031925779476296157}
+{"step": 128, "steps": 250, "lr": 2.931049779612634e-05, "grad_norm": 3.751028060913086, "loss_scale": 4.0, "retries": 0, "seconds": 161.5, "max_memory_gb": 17.17, "ce": 0.13909340541431447, "kl": 0.5047681655123597, "context": 0.3914775028824806}
+{"step": 129, "steps": 250, "lr": 2.9009246998797557e-05, "grad_norm": 1.3618165254592896, "loss_scale": 4.0, "retries": 0, "seconds": 162.8, "max_memory_gb": 17.17, "ce": 0.12073166645132005, "kl": 0.04012799424890545, "context": 0.1407956581970211}
+{"step": 130, "steps": 250, "lr": 2.8707724168320238e-05, "grad_norm": 0.709721565246582, "loss_scale": 4.0, "retries": 0, "seconds": 164.1, "max_memory_gb": 17.17, "ce": 0.040449977037496865, "kl": 0.31204834071104415, "context": 0.196474147727713}
+{"step": 131, "steps": 250, "lr": 2.840598365246184e-05, "grad_norm": 6.008906841278076, "loss_scale": 4.0, "retries": 0, "seconds": 165.4, "max_memory_gb": 17.17, "ce": 0.29347816860536113, "kl": 0.3201913292723475, "context": 0.45357383077498525}
+{"step": 132, "steps": 250, "lr": 2.8104079838226383e-05, "grad_norm": 1.4746735095977783, "loss_scale": 4.0, "retries": 0, "seconds": 166.6, "max_memory_gb": 17.17, "ce": 0.20136970520252362, "kl": 0.03481296145764645, "context": 0.21877619056613185}
+{"step": 133, "steps": 250, "lr": 2.7802067142051446e-05, "grad_norm": 1.5432138442993164, "loss_scale": 4.0, "retries": 0, "seconds": 167.9, "max_memory_gb": 17.17, "ce": 0.28756557727683685, "kl": 0.013819290732499212, "context": 0.2944752206094563}
+{"step": 134, "steps": 250, "lr": 2.7500000000000004e-05, "grad_norm": 1.356660008430481, "loss_scale": 4.0, "retries": 0, "seconds": 169.2, "max_memory_gb": 17.17, "ce": 0.19810728916490916, "kl": 0.0038620750710833818, "context": 0.20003833346709143}
+{"step": 135, "steps": 250, "lr": 2.7197932857948566e-05, "grad_norm": 0.5119791626930237, "loss_scale": 4.0, "retries": 0, "seconds": 170.3, "max_memory_gb": 17.17, "ce": 0.015109694737475365, "kl": 0.04026095608060132, "context": 0.035240173659985885}
+{"step": 136, "steps": 250, "lr": 2.689592016177363e-05, "grad_norm": 3.8460733890533447, "loss_scale": 4.0, "retries": 0, "seconds": 171.7, "max_memory_gb": 17.17, "ce": 0.09625448752194643, "kl": 0.1304368746059481, "context": 0.16147292510140687}
+{"step": 137, "steps": 250, "lr": 2.659401634753817e-05, "grad_norm": 0.7498811483383179, "loss_scale": 4.0, "retries": 0, "seconds": 172.8, "max_memory_gb": 17.17, "ce": 0.014793147973250598, "kl": 0.03857260233053239, "context": 0.034079449163982645}
+{"step": 138, "steps": 250, "lr": 2.6292275831679764e-05, "grad_norm": 2.7616779804229736, "loss_scale": 4.0, "retries": 0, "seconds": 174.0, "max_memory_gb": 17.17, "ce": 0.09521973156370223, "kl": 0.15401718579232693, "context": 0.17222832422703505}
+{"step": 139, "steps": 250, "lr": 2.5990753001202445e-05, "grad_norm": 1.5320258140563965, "loss_scale": 4.0, "retries": 0, "seconds": 175.3, "max_memory_gb": 17.17, "ce": 0.010559653470409103, "kl": 0.14003665479367555, "context": 0.08057797871879302}
+{"step": 140, "steps": 250, "lr": 2.5689502203873672e-05, "grad_norm": 0.08961468189954758, "loss_scale": 4.0, "retries": 0, "seconds": 176.5, "max_memory_gb": 17.17, "ce": 0.0054258227901300415, "kl": 0.002171272979467176, "context": 0.0065114593744510785}
+{"step": 141, "steps": 250, "lr": 2.538857773842847e-05, "grad_norm": 0.09061591327190399, "loss_scale": 4.0, "retries": 0, "seconds": 177.8, "max_memory_gb": 17.17, "ce": 0.007736893538094591, "kl": 0.0008616399918537354, "context": 0.00816771372774383}
+{"step": 142, "steps": 250, "lr": 2.5088033844782426e-05, "grad_norm": 1.5224192142486572, "loss_scale": 4.0, "retries": 0, "seconds": 179.1, "max_memory_gb": 17.17, "ce": 0.06642447400372475, "kl": 0.013287052104715258, "context": 0.0730680018896237}
+{"step": 143, "steps": 250, "lr": 2.4787924694255237e-05, "grad_norm": 4.493057727813721, "loss_scale": 4.0, "retries": 0, "seconds": 180.3, "max_memory_gb": 17.17, "ce": 0.40626950447040144, "kl": 0.0582815574198321, "context": 0.43541027925675735}
+{"step": 144, "steps": 250, "lr": 2.4488304379806627e-05, "grad_norm": 0.7256802320480347, "loss_scale": 4.0, "retries": 0, "seconds": 181.5, "max_memory_gb": 17.17, "ce": 0.023731307650450617, "kl": 0.08904628924210556, "context": 0.06825445085996762}
+{"step": 145, "steps": 250, "lr": 2.4189226906286388e-05, "grad_norm": 0.0387493334710598, "loss_scale": 4.0, "retries": 0, "seconds": 182.8, "max_memory_gb": 17.17, "ce": 0.0016644204879412428, "kl": 0.003316356179311697, "context": 0.003322598640806973}
+{"step": 146, "steps": 250, "lr": 2.3890746180700395e-05, "grad_norm": 0.28587061166763306, "loss_scale": 4.0, "retries": 0, "seconds": 184.0, "max_memory_gb": 17.17, "ce": 0.017424855584977195, "kl": 0.015998158531147055, "context": 0.02542393541079946}
+{"step": 147, "steps": 250, "lr": 2.359291600249407e-05, "grad_norm": 2.255601167678833, "loss_scale": 4.0, "retries": 0, "seconds": 185.3, "max_memory_gb": 17.17, "ce": 0.37957466034276877, "kl": 0.3723091245192336, "context": 0.5657292183896061}
+{"step": 148, "steps": 250, "lr": 2.3295790053855392e-05, "grad_norm": 0.28373411297798157, "loss_scale": 4.0, "retries": 0, "seconds": 186.5, "max_memory_gb": 17.17, "ce": 0.013384657344431616, "kl": 0.009516864651232027, "context": 0.01814308960456401}
+{"step": 149, "steps": 250, "lr": 2.299942189003901e-05, "grad_norm": 0.16595110297203064, "loss_scale": 4.0, "retries": 0, "seconds": 187.7, "max_memory_gb": 17.17, "ce": 0.014109465351793915, "kl": 0.0020955449243729163, "context": 0.015157237503444776}
+{"step": 150, "steps": 250, "lr": 2.2703864929713144e-05, "grad_norm": 0.6490943431854248, "loss_scale": 4.0, "retries": 0, "seconds": 189.0, "max_memory_gb": 17.17, "ce": 0.07032521559449378, "kl": 0.02888543233711971, "context": 0.08476792986039072}
+{"step": 151, "steps": 250, "lr": 2.2409172445331257e-05, "grad_norm": 1.0086989402770996, "loss_scale": 4.0, "retries": 0, "seconds": 190.2, "max_memory_gb": 17.17, "ce": 0.032183982388232835, "kl": 0.044965166205656715, "context": 0.054666563635692}
+{"step": 152, "steps": 250, "lr": 2.211539755352996e-05, "grad_norm": 1.528773546218872, "loss_scale": 4.0, "retries": 0, "seconds": 191.2, "max_memory_gb": 17.17, "ce": 0.002914762997534126, "kl": 0.2390784264807735, "context": 0.1224539796821773}
+{"step": 153, "steps": 250, "lr": 2.1822593205555016e-05, "grad_norm": 0.7883389592170715, "loss_scale": 4.0, "retries": 0, "seconds": 192.5, "max_memory_gb": 17.17, "ce": 0.006445360864745453, "kl": 0.465800438369115, "context": 0.23934558576729614}
+{"step": 154, "steps": 250, "lr": 2.1530812177717236e-05, "grad_norm": 0.51702880859375, "loss_scale": 4.0, "retries": 0, "seconds": 193.7, "max_memory_gb": 17.17, "ce": 0.027095424055005424, "kl": 0.07567163099884056, "context": 0.064931239845464}
+{"step": 155, "steps": 250, "lr": 2.1240107061879815e-05, "grad_norm": 1.7935001850128174, "loss_scale": 4.0, "retries": 0, "seconds": 194.9, "max_memory_gb": 17.17, "ce": 0.23827658453956246, "kl": 0.008990331132736173, "context": 0.2427717510581715}
+{"step": 156, "steps": 250, "lr": 2.095053025597896e-05, "grad_norm": 2.289109230041504, "loss_scale": 4.0, "retries": 0, "seconds": 196.2, "max_memory_gb": 17.17, "ce": 0.023673482137382962, "kl": 0.019596283299279094, "context": 0.03347162374120671}
+{"step": 157, "steps": 250, "lr": 2.066213395457945e-05, "grad_norm": 0.5718783736228943, "loss_scale": 4.0, "retries": 0, "seconds": 197.4, "max_memory_gb": 17.17, "ce": 0.0294810535851866, "kl": 0.03772714272054145, "context": 0.04834462492726743}
+{"step": 158, "steps": 250, "lr": 2.0374970139466877e-05, "grad_norm": 2.07983660697937, "loss_scale": 4.0, "retries": 0, "seconds": 198.6, "max_memory_gb": 17.17, "ce": 0.21973309095483273, "kl": 0.12414086961143767, "context": 0.2818035229865927}
+{"step": 159, "steps": 250, "lr": 2.0089090570278213e-05, "grad_norm": 1.0751720666885376, "loss_scale": 4.0, "retries": 0, "seconds": 199.9, "max_memory_gb": 17.17, "ce": 0.008693620038684458, "kl": 0.06388182344380766, "context": 0.04063453106209636}
+{"step": 160, "steps": 250, "lr": 1.9804546775172457e-05, "grad_norm": 1.4148929119110107, "loss_scale": 4.0, "retries": 0, "seconds": 201.1, "max_memory_gb": 17.17, "ce": 0.06081459764391184, "kl": 0.2809963685940602, "context": 0.20131277909968048}
+{"step": 161, "steps": 250, "lr": 1.9521390041542946e-05, "grad_norm": 1.5704433917999268, "loss_scale": 4.0, "retries": 0, "seconds": 202.4, "max_memory_gb": 17.17, "ce": 0.14151818995014764, "kl": 0.09525968076104618, "context": 0.1891480368503835}
+{"step": 162, "steps": 250, "lr": 1.923967140677313e-05, "grad_norm": 2.592381000518799, "loss_scale": 4.0, "retries": 0, "seconds": 203.6, "max_memory_gb": 17.17, "ce": 0.10634374723304063, "kl": 0.15187867251006537, "context": 0.1822830834717024}
+{"step": 163, "steps": 250, "lr": 1.8959441649037335e-05, "grad_norm": 6.787980079650879, "loss_scale": 4.0, "retries": 0, "seconds": 204.8, "max_memory_gb": 17.17, "ce": 0.23795868421439081, "kl": 0.6963384506998409, "context": 0.5861279040982481}
+{"step": 164, "steps": 250, "lr": 1.868075127814832e-05, "grad_norm": 5.730381011962891, "loss_scale": 4.0, "retries": 0, "seconds": 206.1, "max_memory_gb": 17.17, "ce": 0.11424376833019778, "kl": 0.09104886052409711, "context": 0.1597681968996767}
+{"step": 165, "steps": 250, "lr": 1.8403650526453143e-05, "grad_norm": 0.24014680087566376, "loss_scale": 4.0, "retries": 0, "seconds": 207.3, "max_memory_gb": 17.17, "ce": 0.010642519206157885, "kl": 0.008589693272369914, "context": 0.01493736557313241}
+{"step": 166, "steps": 250, "lr": 1.812818933977912e-05, "grad_norm": 3.0271379947662354, "loss_scale": 4.0, "retries": 0, "seconds": 208.5, "max_memory_gb": 17.17, "ce": 0.3356161516858265, "kl": 0.056548596345237456, "context": 0.3638904549879953}
+{"step": 167, "steps": 250, "lr": 1.785441736843128e-05, "grad_norm": 0.416531503200531, "loss_scale": 4.0, "retries": 0, "seconds": 209.8, "max_memory_gb": 17.17, "ce": 0.023481833311961964, "kl": 0.007056390924844891, "context": 0.02701002883259207}
+{"step": 168, "steps": 250, "lr": 1.7582383958243314e-05, "grad_norm": 0.15441541373729706, "loss_scale": 4.0, "retries": 0, "seconds": 211.0, "max_memory_gb": 17.17, "ce": 0.011015054566087201, "kl": 0.001585919912031386, "context": 0.011808014365669806}
+{"step": 169, "steps": 250, "lr": 1.7312138141683164e-05, "grad_norm": 2.1052916049957275, "loss_scale": 4.0, "retries": 0, "seconds": 212.3, "max_memory_gb": 17.17, "ce": 0.07783450493298005, "kl": 0.9295248929411173, "context": 0.5425969497300684}
+{"step": 170, "steps": 250, "lr": 1.704372862901521e-05, "grad_norm": 0.012080008164048195, "loss_scale": 4.0, "retries": 0, "seconds": 213.5, "max_memory_gb": 17.17, "ce": 0.000836640436318703, "kl": 7.070618903526338e-05, "context": 0.0008719935285625979}
+{"step": 171, "steps": 250, "lr": 1.6777203799520573e-05, "grad_norm": 0.6412476897239685, "loss_scale": 4.0, "retries": 0, "seconds": 214.5, "max_memory_gb": 17.17, "ce": 0.01793369340157369, "kl": 0.04699471272761002, "context": 0.041431049117818475}
+{"step": 172, "steps": 250, "lr": 1.6512611692777008e-05, "grad_norm": 0.14922253787517548, "loss_scale": 4.0, "retries": 0, "seconds": 215.6, "max_memory_gb": 17.17, "ce": 0.007059664829284884, "kl": 0.012000800707028247, "context": 0.013060065190074965}
+{"step": 173, "steps": 250, "lr": 1.6250000000000005e-05, "grad_norm": 0.2578035295009613, "loss_scale": 4.0, "retries": 0, "seconds": 216.6, "max_memory_gb": 17.17, "ce": 0.014688749855849892, "kl": 0.003036729867744725, "context": 0.01620711456052959}
+{"step": 174, "steps": 250, "lr": 1.598941605544678e-05, "grad_norm": 0.5836048722267151, "loss_scale": 4.0, "retries": 0, "seconds": 217.6, "max_memory_gb": 17.17, "ce": 0.03989370014460292, "kl": 0.011988497037464185, "context": 0.04588794863957446}
+{"step": 175, "steps": 250, "lr": 1.5730906827884506e-05, "grad_norm": 3.9587395191192627, "loss_scale": 4.0, "retries": 0, "seconds": 218.7, "max_memory_gb": 17.17, "ce": 0.1859545619226992, "kl": 0.06946829735534266, "context": 0.22068871650844812}
+{"step": 176, "steps": 250, "lr": 1.5474518912124476e-05, "grad_norm": 0.0625896155834198, "loss_scale": 4.0, "retries": 0, "seconds": 219.7, "max_memory_gb": 17.17, "ce": 0.0041040384385269135, "kl": 0.0007705238385824487, "context": 0.004489300335990265}
+{"step": 177, "steps": 250, "lr": 1.5220298520623646e-05, "grad_norm": 1.1483532190322876, "loss_scale": 4.0, "retries": 0, "seconds": 220.9, "max_memory_gb": 17.17, "ce": 0.04186472022411181, "kl": 0.6799721538554877, "context": 0.3818507869000314}
+{"step": 178, "steps": 250, "lr": 1.4968291475155121e-05, "grad_norm": 1.7212611436843872, "loss_scale": 4.0, "retries": 0, "seconds": 222.2, "max_memory_gb": 17.17, "ce": 0.10515452979598194, "kl": 0.03685815187782282, "context": 0.12358360539656132}
+{"step": 179, "steps": 250, "lr": 1.4718543198548998e-05, "grad_norm": 1.5531973838806152, "loss_scale": 4.0, "retries": 0, "seconds": 223.5, "max_memory_gb": 17.17, "ce": 0.37427887963713147, "kl": 0.01310565627500182, "context": 0.3808316928916611}
+{"step": 180, "steps": 250, "lr": 1.4471098706505227e-05, "grad_norm": 0.5821088552474976, "loss_scale": 4.0, "retries": 0, "seconds": 224.9, "max_memory_gb": 17.17, "ce": 0.0369624648737954, "kl": 0.025317452771560056, "context": 0.049621191719779745}
+{"step": 181, "steps": 250, "lr": 1.4226002599479743e-05, "grad_norm": 0.9128147959709167, "loss_scale": 4.0, "retries": 0, "seconds": 226.1, "max_memory_gb": 17.17, "ce": 0.003094420870183967, "kl": 0.155144503012707, "context": 0.08066667130333371}
+{"step": 182, "steps": 250, "lr": 1.3983299054645471e-05, "grad_norm": 2.2278881072998047, "loss_scale": 4.0, "retries": 0, "seconds": 227.3, "max_memory_gb": 17.17, "ce": 0.0711644621915184, "kl": 0.1950797373747264, "context": 0.16870433039730415}
+{"step": 183, "steps": 250, "lr": 1.3743031817929724e-05, "grad_norm": 0.033490560948848724, "loss_scale": 4.0, "retries": 0, "seconds": 228.8, "max_memory_gb": 17.17, "ce": 0.0021476483234437183, "kl": 0.0002472653886798071, "context": 0.0022712810241500847}
+{"step": 184, "steps": 250, "lr": 1.3505244196129164e-05, "grad_norm": 0.10451270639896393, "loss_scale": 4.0, "retries": 0, "seconds": 230.1, "max_memory_gb": 17.17, "ce": 0.0031521757773589343, "kl": 0.003901779928128235, "context": 0.005103065748699009}
+{"step": 185, "steps": 250, "lr": 1.3269979049104014e-05, "grad_norm": 0.6128019094467163, "loss_scale": 4.0, "retries": 0, "seconds": 231.4, "max_memory_gb": 17.17, "ce": 0.0063700760510982946, "kl": 0.11210427963123948, "context": 0.062422215996775776}
+{"step": 186, "steps": 250, "lr": 1.3037278782052863e-05, "grad_norm": 5.548933506011963, "loss_scale": 4.0, "retries": 0, "seconds": 232.7, "max_memory_gb": 17.17, "ce": 0.0780483673006529, "kl": 0.12614889635005966, "context": 0.1411228197539458}
+{"step": 187, "steps": 250, "lr": 1.2807185337869343e-05, "grad_norm": 1.023084044456482, "loss_scale": 4.0, "retries": 0, "seconds": 233.9, "max_memory_gb": 17.17, "ce": 0.01680076215416193, "kl": 0.5696621932147536, "context": 0.30163184528646525}
+{"step": 188, "steps": 250, "lr": 1.2579740189582115e-05, "grad_norm": 1.8572204113006592, "loss_scale": 4.0, "retries": 0, "seconds": 235.2, "max_memory_gb": 17.17, "ce": 0.22775245810043998, "kl": 0.007372004978606128, "context": 0.23143845732556656}
+{"step": 189, "steps": 250, "lr": 1.2354984332879668e-05, "grad_norm": 4.497631549835205, "loss_scale": 4.0, "retries": 0, "seconds": 236.6, "max_memory_gb": 17.17, "ce": 0.3661491879756795, "kl": 0.230597198176838, "context": 0.4814477833569981}
+{"step": 190, "steps": 250, "lr": 1.2132958278721064e-05, "grad_norm": 0.036896634846925735, "loss_scale": 4.0, "retries": 0, "seconds": 237.9, "max_memory_gb": 17.17, "ce": 0.0015686947881476954, "kl": 0.0016930039710132405, "context": 0.002415196708170697}
+{"step": 191, "steps": 250, "lr": 1.1913702046034016e-05, "grad_norm": 7.261673927307129, "loss_scale": 4.0, "retries": 0, "seconds": 239.1, "max_memory_gb": 17.17, "ce": 0.19988068072416354, "kl": 0.14114500339383085, "context": 0.270453175122384}
+{"step": 192, "steps": 250, "lr": 1.1697255154501756e-05, "grad_norm": 1.6089158058166504, "loss_scale": 4.0, "retries": 0, "seconds": 240.5, "max_memory_gb": 17.17, "ce": 0.39498102557263337, "kl": 0.05040275424471474, "context": 0.42018239058961626}
+{"step": 193, "steps": 250, "lr": 1.1483656617439842e-05, "grad_norm": 0.05588650703430176, "loss_scale": 4.0, "retries": 0, "seconds": 241.7, "max_memory_gb": 17.17, "ce": 0.0030913303489796817, "kl": 0.0016485418200318236, "context": 0.003915601366315968}
+{"step": 194, "steps": 250, "lr": 1.1272944934764177e-05, "grad_norm": 0.5697285532951355, "loss_scale": 4.0, "retries": 0, "seconds": 243.0, "max_memory_gb": 17.17, "ce": 0.06516207725508139, "kl": 0.008790803079136822, "context": 0.06955747865140438}
+{"step": 195, "steps": 250, "lr": 1.1065158086051684e-05, "grad_norm": 0.1802186816930771, "loss_scale": 4.0, "retries": 0, "seconds": 244.2, "max_memory_gb": 17.17, "ce": 0.016408213297836483, "kl": 0.0019357281908014556, "context": 0.017376077143126167}
+{"step": 196, "steps": 250, "lr": 1.086033352369467e-05, "grad_norm": 1.1190999746322632, "loss_scale": 4.0, "retries": 0, "seconds": 245.4, "max_memory_gb": 17.17, "ce": 0.0031551385109196417, "kl": 0.18181666443638278, "context": 0.09406347355252365}
+{"step": 197, "steps": 250, "lr": 1.0658508166150224e-05, "grad_norm": 2.535748243331909, "loss_scale": 4.0, "retries": 0, "seconds": 246.7, "max_memory_gb": 17.17, "ce": 0.03175850534171332, "kl": 0.6040386094509813, "context": 0.3337778112036176}
+{"step": 198, "steps": 250, "lr": 1.0459718391285954e-05, "grad_norm": 0.6560767889022827, "loss_scale": 4.0, "retries": 0, "seconds": 248.0, "max_memory_gb": 17.17, "ce": 0.08311339264764683, "kl": 0.00563709829657455, "context": 0.08593194351124112}
+{"step": 199, "steps": 250, "lr": 1.0264000029822999e-05, "grad_norm": 1.3852609395980835, "loss_scale": 4.0, "retries": 0, "seconds": 249.2, "max_memory_gb": 17.17, "ce": 0.10748330192291178, "kl": 0.006187355622387258, "context": 0.11057697719661519}
+{"step": 200, "steps": 250, "lr": 1.0071388358877775e-05, "grad_norm": 1.1691035032272339, "loss_scale": 4.0, "retries": 0, "seconds": 250.5, "max_memory_gb": 17.17, "ce": 0.2569088893942535, "kl": 0.025564243667759, "context": 0.26969101815484464}
+{"step": 201, "steps": 250, "lr": 9.881918095603501e-06, "grad_norm": 1.4565809965133667, "loss_scale": 4.0, "retries": 0, "seconds": 251.7, "max_memory_gb": 17.17, "ce": 0.06679193730815314, "kl": 0.08259349400668725, "context": 0.1080886842100881}
+{"step": 202, "steps": 250, "lr": 9.695623390932612e-06, "grad_norm": 0.0651526004076004, "loss_scale": 4.0, "retries": 0, "seconds": 253.0, "max_memory_gb": 17.17, "ce": 0.0030603248451370746, "kl": 0.0030145848415941146, "context": 0.004567617288557813}
+{"step": 203, "steps": 250, "lr": 9.51253782342123e-06, "grad_norm": 0.7104941010475159, "loss_scale": 4.0, "retries": 0, "seconds": 254.3, "max_memory_gb": 17.17, "ce": 0.021725309394241776, "kl": 0.023962804436450824, "context": 0.03370671155425953}
+{"step": 204, "steps": 250, "lr": 9.332694393196846e-06, "grad_norm": 0.24712467193603516, "loss_scale": 4.0, "retries": 0, "seconds": 255.4, "max_memory_gb": 17.17, "ce": 0.01943131088046357, "kl": 0.00278777361381799, "context": 0.020825197861995548}
+{"step": 205, "steps": 250, "lr": 9.156125516010256e-06, "grad_norm": 1.2255409955978394, "loss_scale": 4.0, "retries": 0, "seconds": 256.7, "max_memory_gb": 17.17, "ce": 0.005524724809220061, "kl": 0.3874871596672165, "context": 0.19926829934411217}
+{"step": 206, "steps": 250, "lr": 8.982863017392733e-06, "grad_norm": 0.09907635301351547, "loss_scale": 4.0, "retries": 0, "seconds": 257.9, "max_memory_gb": 17.17, "ce": 0.004652115938370116, "kl": 0.002619865351334738, "context": 0.005962048613582738}
+{"step": 207, "steps": 250, "lr": 8.81293812691974e-06, "grad_norm": 0.5766837000846863, "loss_scale": 4.0, "retries": 0, "seconds": 259.1, "max_memory_gb": 17.17, "ce": 0.07968358125071973, "kl": 0.06289727403782308, "context": 0.11113221466075629}
+{"step": 208, "steps": 250, "lr": 8.646381472581927e-06, "grad_norm": 0.06319650262594223, "loss_scale": 4.0, "retries": 0, "seconds": 260.5, "max_memory_gb": 17.17, "ce": 0.0036759084468940273, "kl": 0.0010537359758018283, "context": 0.004202776530291885}
+{"step": 209, "steps": 250, "lr": 8.483223075264622e-06, "grad_norm": 2.1961748600006104, "loss_scale": 4.0, "retries": 0, "seconds": 261.7, "max_memory_gb": 17.17, "ce": 0.20682186711928807, "kl": 0.049354431190295145, "context": 0.23149908031336963}
+{"step": 210, "steps": 250, "lr": 8.323492343336747e-06, "grad_norm": 1.674338459968567, "loss_scale": 4.0, "retries": 0, "seconds": 262.9, "max_memory_gb": 17.17, "ce": 0.43754483936936595, "kl": 0.09530429942060437, "context": 0.4851969888986787}
+{"step": 211, "steps": 250, "lr": 8.167218067350122e-06, "grad_norm": 0.6273200511932373, "loss_scale": 4.0, "retries": 0, "seconds": 264.1, "max_memory_gb": 17.17, "ce": 0.12835876308963634, "kl": 0.06510236650501611, "context": 0.16090994645492174}
+{"step": 212, "steps": 250, "lr": 8.01442841485013e-06, "grad_norm": 3.747847318649292, "loss_scale": 4.0, "retries": 0, "seconds": 265.2, "max_memory_gb": 17.17, "ce": 0.09582464980485383, "kl": 0.880074037333543, "context": 0.5358616698649712}
+{"step": 213, "steps": 250, "lr": 7.865150925298678e-06, "grad_norm": 0.9482322335243225, "loss_scale": 4.0, "retries": 0, "seconds": 266.5, "max_memory_gb": 17.17, "ce": 0.18932945927372202, "kl": 0.026920646416328964, "context": 0.20278978282294702}
+{"step": 214, "steps": 250, "lr": 7.719412505110375e-06, "grad_norm": 1.8376635313034058, "loss_scale": 4.0, "retries": 0, "seconds": 267.6, "max_memory_gb": 17.17, "ce": 0.29791262495564297, "kl": 0.06870259519200772, "context": 0.33226391475182027}
+{"step": 215, "steps": 250, "lr": 7.57723942280278e-06, "grad_norm": 2.6459054946899414, "loss_scale": 4.0, "retries": 0, "seconds": 268.7, "max_memory_gb": 17.17, "ce": 0.4859795652446337, "kl": 0.01734326937332753, "context": 0.49465121401590295}
+{"step": 216, "steps": 250, "lr": 7.438657304261687e-06, "grad_norm": 0.8080787062644958, "loss_scale": 4.0, "retries": 0, "seconds": 270.1, "max_memory_gb": 17.17, "ce": 0.02699060903978534, "kl": 0.0825907831240329, "context": 0.06828599923755974}
+{"step": 217, "steps": 250, "lr": 7.30369112812219e-06, "grad_norm": 0.05182049050927162, "loss_scale": 4.0, "retries": 0, "seconds": 271.3, "max_memory_gb": 17.17, "ce": 0.003222625920898281, "kl": 0.0014580361257685581, "context": 0.003951644001062959}
+{"step": 218, "steps": 250, "lr": 7.172365221266403e-06, "grad_norm": 1.567488193511963, "loss_scale": 4.0, "retries": 0, "seconds": 272.5, "max_memory_gb": 17.17, "ce": 0.18011540532461368, "kl": 0.21355291083455086, "context": 0.2868918552994728}
+{"step": 219, "steps": 250, "lr": 7.0447032544387154e-06, "grad_norm": 1.2131296396255493, "loss_scale": 4.0, "retries": 0, "seconds": 273.8, "max_memory_gb": 17.17, "ce": 0.012568645033752546, "kl": 0.17917684111307608, "context": 0.1021570639568381}
+{"step": 220, "steps": 250, "lr": 6.920728237979274e-06, "grad_norm": 0.3098156452178955, "loss_scale": 4.0, "retries": 0, "seconds": 275.0, "max_memory_gb": 17.17, "ce": 0.02738027466693893, "kl": 0.002473530865245266, "context": 0.028617040428798646}
+{"step": 221, "steps": 250, "lr": 6.800462517676456e-06, "grad_norm": 0.6511226296424866, "loss_scale": 4.0, "retries": 0, "seconds": 276.2, "max_memory_gb": 17.17, "ce": 0.002986372863233555, "kl": 0.06680867831528303, "context": 0.036390711640706286}
+{"step": 222, "steps": 250, "lr": 6.683927770739222e-06, "grad_norm": 1.0944263935089111, "loss_scale": 4.0, "retries": 0, "seconds": 277.3, "max_memory_gb": 17.17, "ce": 0.0501623532618396, "kl": 0.07006537762219978, "context": 0.08519504420110025}
+{"step": 223, "steps": 250, "lr": 6.571145001889892e-06, "grad_norm": 3.1428110599517822, "loss_scale": 4.0, "retries": 0, "seconds": 278.3, "max_memory_gb": 17.17, "ce": 0.1294359092717059, "kl": 0.0060763852488889825, "context": 0.13247409884934314}
+{"step": 224, "steps": 250, "lr": 6.46213453957817e-06, "grad_norm": 2.5491209030151367, "loss_scale": 4.0, "retries": 0, "seconds": 279.3, "max_memory_gb": 17.17, "ce": 0.06842046650126576, "kl": 0.05881344243971398, "context": 0.09782718756468967}
+{"step": 225, "steps": 250, "lr": 6.356916032317063e-06, "grad_norm": 0.1873820275068283, "loss_scale": 4.0, "retries": 0, "seconds": 280.6, "max_memory_gb": 17.17, "ce": 0.005734621925512329, "kl": 0.016332834435161203, "context": 0.013901039405027404}
+{"step": 226, "steps": 250, "lr": 6.255508445141349e-06, "grad_norm": 0.9641285538673401, "loss_scale": 4.0, "retries": 0, "seconds": 281.8, "max_memory_gb": 17.17, "ce": 0.03704295522766188, "kl": 0.011887751490576193, "context": 0.04298683101660572}
+{"step": 227, "steps": 250, "lr": 6.157930056189228e-06, "grad_norm": 0.516386866569519, "loss_scale": 4.0, "retries": 0, "seconds": 283.2, "max_memory_gb": 17.17, "ce": 0.019972269525169395, "kl": 0.00916302411678771, "context": 0.024553782044677064}
+{"step": 228, "steps": 250, "lr": 6.064198453407796e-06, "grad_norm": 0.2901771664619446, "loss_scale": 4.0, "retries": 0, "seconds": 284.4, "max_memory_gb": 17.17, "ce": 0.009484111236815806, "kl": 0.023958671605214477, "context": 0.021463446551933885}
+{"step": 229, "steps": 250, "lr": 5.974330531382935e-06, "grad_norm": 2.908763885498047, "loss_scale": 4.0, "retries": 0, "seconds": 285.6, "max_memory_gb": 17.17, "ce": 0.08379159771720879, "kl": 0.04133086406545772, "context": 0.10445703045115806}
+{"step": 230, "steps": 250, "lr": 5.888342488294125e-06, "grad_norm": 3.1589622497558594, "loss_scale": 4.0, "retries": 0, "seconds": 287.0, "max_memory_gb": 17.17, "ce": 0.42261693527689204, "kl": 0.27885688113747165, "context": 0.562045359169133}
+{"step": 231, "steps": 250, "lr": 5.806249822994849e-06, "grad_norm": 0.5920528173446655, "loss_scale": 4.0, "retries": 0, "seconds": 288.3, "max_memory_gb": 17.17, "ce": 0.04018870272557251, "kl": 0.07059471854881849, "context": 0.07548606270574965}
+{"step": 232, "steps": 250, "lr": 5.728067332218994e-06, "grad_norm": 0.5211964845657349, "loss_scale": 4.0, "retries": 0, "seconds": 289.5, "max_memory_gb": 17.17, "ce": 0.032031909344368614, "kl": 0.008719411642232444, "context": 0.03639161375758704}
+{"step": 233, "steps": 250, "lr": 5.65380910791383e-06, "grad_norm": 0.9141502380371094, "loss_scale": 4.0, "retries": 0, "seconds": 291.0, "max_memory_gb": 17.17, "ce": 0.023848566692322493, "kl": 0.11411690397653729, "context": 0.0809070176910609}
+{"step": 234, "steps": 250, "lr": 5.583488534700022e-06, "grad_norm": 0.6933618783950806, "loss_scale": 4.0, "retries": 0, "seconds": 292.2, "max_memory_gb": 17.17, "ce": 0.01879931171424687, "kl": 0.02921888391210814, "context": 0.03340875363210216}
+{"step": 235, "steps": 250, "lr": 5.517118287459118e-06, "grad_norm": 2.3718132972717285, "loss_scale": 4.0, "retries": 0, "seconds": 293.7, "max_memory_gb": 17.17, "ce": 0.30445248921751045, "kl": 0.011041322250093799, "context": 0.30997314015985467}
+{"step": 236, "steps": 250, "lr": 5.454710329048978e-06, "grad_norm": 0.23126836121082306, "loss_scale": 4.0, "retries": 0, "seconds": 294.9, "max_memory_gb": 17.17, "ce": 0.01494380718213506, "kl": 0.016528321648365818, "context": 0.023207967868074775}
+{"step": 237, "steps": 250, "lr": 5.3962759081475436e-06, "grad_norm": 1.1856803894042969, "loss_scale": 4.0, "retries": 0, "seconds": 296.2, "max_memory_gb": 17.17, "ce": 0.08149478086852469, "kl": 0.02313030015420736, "context": 0.09305993292946368}
+{"step": 238, "steps": 250, "lr": 5.341825557225321e-06, "grad_norm": 0.4463335871696472, "loss_scale": 4.0, "retries": 0, "seconds": 297.6, "max_memory_gb": 17.17, "ce": 0.026966275669110473, "kl": 0.005733682666686946, "context": 0.029833116481313482}
+{"step": 239, "steps": 250, "lr": 5.291369090646961e-06, "grad_norm": 0.5910401344299316, "loss_scale": 4.0, "retries": 0, "seconds": 298.9, "max_memory_gb": 17.17, "ce": 0.10775129776448011, "kl": 0.012484355458582286, "context": 0.1139934747479856}
+{"step": 240, "steps": 250, "lr": 5.244915602902289e-06, "grad_norm": 0.488700270652771, "loss_scale": 4.0, "retries": 0, "seconds": 300.1, "max_memory_gb": 17.17, "ce": 0.035639672802062705, "kl": 0.03902005686904886, "context": 0.05514970331569202}
+{"step": 241, "steps": 250, "lr": 5.20247346696706e-06, "grad_norm": 1.4611271619796753, "loss_scale": 4.0, "retries": 0, "seconds": 301.6, "max_memory_gb": 17.17, "ce": 0.09635987057117745, "kl": 0.01897296353126876, "context": 0.10584635002305731}
+{"step": 242, "steps": 250, "lr": 5.164050332793787e-06, "grad_norm": 0.18465083837509155, "loss_scale": 4.0, "retries": 0, "seconds": 302.9, "max_memory_gb": 17.17, "ce": 0.01089476901688613, "kl": 0.013611099738682242, "context": 0.01770031900377944}
+{"step": 243, "steps": 250, "lr": 5.129653125932873e-06, "grad_norm": 0.03313751146197319, "loss_scale": 4.0, "retries": 0, "seconds": 304.2, "max_memory_gb": 17.17, "ce": 0.002197605645051226, "kl": 0.0010662813765520696, "context": 0.0027307462878525257}
+{"step": 244, "steps": 250, "lr": 5.099288046284329e-06, "grad_norm": 3.476334571838379, "loss_scale": 4.0, "retries": 0, "seconds": 305.5, "max_memory_gb": 17.17, "ce": 0.03870360165456077, "kl": 0.2231944789245972, "context": 0.15030083902092883}
+{"step": 245, "steps": 250, "lr": 5.072960566980277e-06, "grad_norm": 1.9912056922912598, "loss_scale": 4.0, "retries": 0, "seconds": 306.7, "max_memory_gb": 17.17, "ce": 0.3261501084052725, "kl": 0.03714520546782296, "context": 0.3447227154101711}
+{"step": 246, "steps": 250, "lr": 5.050675433398435e-06, "grad_norm": 0.03206944465637207, "loss_scale": 4.0, "retries": 0, "seconds": 308.1, "max_memory_gb": 17.17, "ce": 0.0017416382324881852, "kl": 0.0006531766248372151, "context": 0.002068226531264372}
+{"step": 247, "steps": 250, "lr": 5.032436662306811e-06, "grad_norm": 1.4456523656845093, "loss_scale": 4.0, "retries": 0, "seconds": 309.4, "max_memory_gb": 17.17, "ce": 0.008376201134524308, "kl": 0.23037876975286053, "context": 0.12356558532337658}
+{"step": 248, "steps": 250, "lr": 5.018247541139684e-06, "grad_norm": 0.3141365349292755, "loss_scale": 4.0, "retries": 0, "seconds": 310.6, "max_memory_gb": 17.17, "ce": 0.021156821865588427, "kl": 0.009835484204813838, "context": 0.02607456431724131}
+{"step": 249, "steps": 250, "lr": 5.00811062740508e-06, "grad_norm": 1.0759930610656738, "loss_scale": 4.0, "retries": 0, "seconds": 312.0, "max_memory_gb": 17.17, "ce": 0.05843111639842391, "kl": 0.05914801196195185, "context": 0.08800512598827481}
+{"step": 250, "steps": 250, "lr": 5.002027748223778e-06, "grad_norm": 0.1959456205368042, "loss_scale": 4.0, "retries": 0, "seconds": 313.2, "max_memory_gb": 17.17, "ce": 0.00991423733648844, "kl": 0.006977918608754408, "context": 0.013403196935541928}

runs/fixed_internvl/context/Random8_8104_s13/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /data/OPSD/assets/models/InternVL3_5-8B-HF
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/data/OPSD/assets/models/InternVL3_5-8B-HF
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

runs/fixed_internvl/context/Random8_8104_s13/adapter/adapter_config.json

@@ -0,0 +1,213 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "model.language_model.layers.16.mlp.down_proj": 144,
+    "model.language_model.layers.16.mlp.gate_proj": 144,
+    "model.language_model.layers.16.mlp.up_proj": 144,
+    "model.language_model.layers.16.self_attn.k_proj": 144,
+    "model.language_model.layers.16.self_attn.o_proj": 144,
+    "model.language_model.layers.16.self_attn.q_proj": 144,
+    "model.language_model.layers.16.self_attn.v_proj": 144,
+    "model.language_model.layers.18.mlp.down_proj": 144,
+    "model.language_model.layers.18.mlp.gate_proj": 144,
+    "model.language_model.layers.18.mlp.up_proj": 144,
+    "model.language_model.layers.18.self_attn.k_proj": 144,
+    "model.language_model.layers.18.self_attn.o_proj": 144,
+    "model.language_model.layers.18.self_attn.q_proj": 144,
+    "model.language_model.layers.18.self_attn.v_proj": 144,
+    "model.language_model.layers.19.mlp.down_proj": 144,
+    "model.language_model.layers.19.mlp.gate_proj": 144,
+    "model.language_model.layers.19.mlp.up_proj": 144,
+    "model.language_model.layers.19.self_attn.k_proj": 144,
+    "model.language_model.layers.19.self_attn.o_proj": 144,
+    "model.language_model.layers.19.self_attn.q_proj": 144,
+    "model.language_model.layers.19.self_attn.v_proj": 144,
+    "model.language_model.layers.2.mlp.down_proj": 144,
+    "model.language_model.layers.2.mlp.gate_proj": 144,
+    "model.language_model.layers.2.mlp.up_proj": 144,
+    "model.language_model.layers.2.self_attn.k_proj": 144,
+    "model.language_model.layers.2.self_attn.o_proj": 144,
+    "model.language_model.layers.2.self_attn.q_proj": 144,
+    "model.language_model.layers.2.self_attn.v_proj": 144,
+    "model.language_model.layers.20.mlp.down_proj": 144,
+    "model.language_model.layers.20.mlp.gate_proj": 144,
+    "model.language_model.layers.20.mlp.up_proj": 144,
+    "model.language_model.layers.20.self_attn.k_proj": 144,
+    "model.language_model.layers.20.self_attn.o_proj": 144,
+    "model.language_model.layers.20.self_attn.q_proj": 144,
+    "model.language_model.layers.20.self_attn.v_proj": 144,
+    "model.language_model.layers.25.mlp.down_proj": 144,
+    "model.language_model.layers.25.mlp.gate_proj": 144,
+    "model.language_model.layers.25.mlp.up_proj": 144,
+    "model.language_model.layers.25.self_attn.k_proj": 144,
+    "model.language_model.layers.25.self_attn.o_proj": 144,
+    "model.language_model.layers.25.self_attn.q_proj": 144,
+    "model.language_model.layers.25.self_attn.v_proj": 144,
+    "model.language_model.layers.27.mlp.down_proj": 144,
+    "model.language_model.layers.27.mlp.gate_proj": 144,
+    "model.language_model.layers.27.mlp.up_proj": 144,
+    "model.language_model.layers.27.self_attn.k_proj": 144,
+    "model.language_model.layers.27.self_attn.o_proj": 144,
+    "model.language_model.layers.27.self_attn.q_proj": 144,
+    "model.language_model.layers.27.self_attn.v_proj": 144,
+    "model.language_model.layers.9.mlp.down_proj": 144,
+    "model.language_model.layers.9.mlp.gate_proj": 144,
+    "model.language_model.layers.9.mlp.up_proj": 144,
+    "model.language_model.layers.9.self_attn.k_proj": 144,
+    "model.language_model.layers.9.self_attn.o_proj": 144,
+    "model.language_model.layers.9.self_attn.q_proj": 144,
+    "model.language_model.layers.9.self_attn.v_proj": 144
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/OPSD/assets/models/InternVL3_5-8B-HF",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {
+    "model.language_model.layers.16.mlp.down_proj": 72,
+    "model.language_model.layers.16.mlp.gate_proj": 72,
+    "model.language_model.layers.16.mlp.up_proj": 72,
+    "model.language_model.layers.16.self_attn.k_proj": 72,
+    "model.language_model.layers.16.self_attn.o_proj": 72,
+    "model.language_model.layers.16.self_attn.q_proj": 72,
+    "model.language_model.layers.16.self_attn.v_proj": 72,
+    "model.language_model.layers.18.mlp.down_proj": 72,
+    "model.language_model.layers.18.mlp.gate_proj": 72,
+    "model.language_model.layers.18.mlp.up_proj": 72,
+    "model.language_model.layers.18.self_attn.k_proj": 72,
+    "model.language_model.layers.18.self_attn.o_proj": 72,
+    "model.language_model.layers.18.self_attn.q_proj": 72,
+    "model.language_model.layers.18.self_attn.v_proj": 72,
+    "model.language_model.layers.19.mlp.down_proj": 72,
+    "model.language_model.layers.19.mlp.gate_proj": 72,
+    "model.language_model.layers.19.mlp.up_proj": 72,
+    "model.language_model.layers.19.self_attn.k_proj": 72,
+    "model.language_model.layers.19.self_attn.o_proj": 72,
+    "model.language_model.layers.19.self_attn.q_proj": 72,
+    "model.language_model.layers.19.self_attn.v_proj": 72,
+    "model.language_model.layers.2.mlp.down_proj": 72,
+    "model.language_model.layers.2.mlp.gate_proj": 72,
+    "model.language_model.layers.2.mlp.up_proj": 72,
+    "model.language_model.layers.2.self_attn.k_proj": 72,
+    "model.language_model.layers.2.self_attn.o_proj": 72,
+    "model.language_model.layers.2.self_attn.q_proj": 72,
+    "model.language_model.layers.2.self_attn.v_proj": 72,
+    "model.language_model.layers.20.mlp.down_proj": 72,
+    "model.language_model.layers.20.mlp.gate_proj": 72,
+    "model.language_model.layers.20.mlp.up_proj": 72,
+    "model.language_model.layers.20.self_attn.k_proj": 72,
+    "model.language_model.layers.20.self_attn.o_proj": 72,
+    "model.language_model.layers.20.self_attn.q_proj": 72,
+    "model.language_model.layers.20.self_attn.v_proj": 72,
+    "model.language_model.layers.25.mlp.down_proj": 72,
+    "model.language_model.layers.25.mlp.gate_proj": 72,
+    "model.language_model.layers.25.mlp.up_proj": 72,
+    "model.language_model.layers.25.self_attn.k_proj": 72,
+    "model.language_model.layers.25.self_attn.o_proj": 72,
+    "model.language_model.layers.25.self_attn.q_proj": 72,
+    "model.language_model.layers.25.self_attn.v_proj": 72,
+    "model.language_model.layers.27.mlp.down_proj": 72,
+    "model.language_model.layers.27.mlp.gate_proj": 72,
+    "model.language_model.layers.27.mlp.up_proj": 72,
+    "model.language_model.layers.27.self_attn.k_proj": 72,
+    "model.language_model.layers.27.self_attn.o_proj": 72,
+    "model.language_model.layers.27.self_attn.q_proj": 72,
+    "model.language_model.layers.27.self_attn.v_proj": 72,
+    "model.language_model.layers.9.mlp.down_proj": 72,
+    "model.language_model.layers.9.mlp.gate_proj": 72,
+    "model.language_model.layers.9.mlp.up_proj": 72,
+    "model.language_model.layers.9.self_attn.k_proj": 72,
+    "model.language_model.layers.9.self_attn.o_proj": 72,
+    "model.language_model.layers.9.self_attn.q_proj": 72,
+    "model.language_model.layers.9.self_attn.v_proj": 72
+  },
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.27.self_attn.k_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.2.mlp.down_proj",
+    "model.language_model.layers.2.self_attn.q_proj",
+    "model.language_model.layers.9.mlp.up_proj",
+    "model.language_model.layers.27.mlp.up_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.25.mlp.gate_proj",
+    "model.language_model.layers.2.self_attn.o_proj",
+    "model.language_model.layers.2.mlp.gate_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.27.self_attn.v_proj",
+    "model.language_model.layers.9.self_attn.k_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.up_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.9.mlp.gate_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.2.self_attn.v_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.27.mlp.gate_proj",
+    "model.language_model.layers.20.self_attn.v_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.25.mlp.down_proj",
+    "model.language_model.layers.9.self_attn.v_proj",
+    "model.language_model.layers.25.self_attn.k_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.27.mlp.down_proj",
+    "model.language_model.layers.9.self_attn.o_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.25.self_attn.v_proj",
+    "model.language_model.layers.27.self_attn.q_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.2.mlp.up_proj",
+    "model.language_model.layers.25.self_attn.o_proj",
+    "model.language_model.layers.2.self_attn.k_proj",
+    "model.language_model.layers.9.mlp.down_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.25.self_attn.q_proj",
+    "model.language_model.layers.27.self_attn.o_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.9.self_attn.q_proj",
+    "model.language_model.layers.16.self_attn.k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}