Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b7bb40e262f2d74d2afb9d16ee6a9be2972c46bdae273dc1cd7bd653fd965b9
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca1b7994149c4f447a8f0637ab1507e45359bc64cee3e069148591746b736f53
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6bebb2ea1f945787716ab50c037f03d76b70074903235532b0543dc9fc4abb19
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ead4cd4b99315ab72203b78b79412374be27509a50aac599e64202914f2c727a
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6e32e20e0d53cef2a6d2f5c69932a577f6085b9274d4217f59da9b5ac723f5e
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/trainer_state.json

@@ -0,0 +1,846 @@
+{
+  "best_metric": 0.4137004315853119,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151",
+  "epoch": 0.9995657837603127,
+  "eval_steps": 10,
+  "global_step": 1151,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008684324793747287,
+      "grad_norm": 1.1732063293457031,
+      "learning_rate": 0.0002,
+      "loss": 1.7067,
+      "step": 10
+    },
+    {
+      "epoch": 0.017368649587494574,
+      "grad_norm": 1.7461568117141724,
+      "learning_rate": 0.0002,
+      "loss": 1.1596,
+      "step": 20
+    },
+    {
+      "epoch": 0.026052974381241857,
+      "grad_norm": 1.1278692483901978,
+      "learning_rate": 0.0002,
+      "loss": 0.9883,
+      "step": 30
+    },
+    {
+      "epoch": 0.03473729917498915,
+      "grad_norm": 1.2258220911026,
+      "learning_rate": 0.0002,
+      "loss": 0.9004,
+      "step": 40
+    },
+    {
+      "epoch": 0.04342162396873643,
+      "grad_norm": 1.0578727722167969,
+      "learning_rate": 0.0002,
+      "loss": 0.7675,
+      "step": 50
+    },
+    {
+      "epoch": 0.052105948762483714,
+      "grad_norm": 1.060376763343811,
+      "learning_rate": 0.0002,
+      "loss": 0.7039,
+      "step": 60
+    },
+    {
+      "epoch": 0.060790273556231005,
+      "grad_norm": 0.9346579313278198,
+      "learning_rate": 0.0002,
+      "loss": 0.7107,
+      "step": 70
+    },
+    {
+      "epoch": 0.0694745983499783,
+      "grad_norm": 0.7925026416778564,
+      "learning_rate": 0.0002,
+      "loss": 0.6864,
+      "step": 80
+    },
+    {
+      "epoch": 0.07815892314372558,
+      "grad_norm": 0.7674112319946289,
+      "learning_rate": 0.0002,
+      "loss": 0.6753,
+      "step": 90
+    },
+    {
+      "epoch": 0.08684324793747286,
+      "grad_norm": 0.8018007278442383,
+      "learning_rate": 0.0002,
+      "loss": 0.628,
+      "step": 100
+    },
+    {
+      "epoch": 0.09552757273122015,
+      "grad_norm": 0.8229703903198242,
+      "learning_rate": 0.0002,
+      "loss": 0.6552,
+      "step": 110
+    },
+    {
+      "epoch": 0.10421189752496743,
+      "grad_norm": 1.0095752477645874,
+      "learning_rate": 0.0002,
+      "loss": 0.6256,
+      "step": 120
+    },
+    {
+      "epoch": 0.11289622231871473,
+      "grad_norm": 1.26262629032135,
+      "learning_rate": 0.0002,
+      "loss": 0.6208,
+      "step": 130
+    },
+    {
+      "epoch": 0.12158054711246201,
+      "grad_norm": 0.6695842742919922,
+      "learning_rate": 0.0002,
+      "loss": 0.6167,
+      "step": 140
+    },
+    {
+      "epoch": 0.13026487190620928,
+      "grad_norm": 0.8266357779502869,
+      "learning_rate": 0.0002,
+      "loss": 0.6326,
+      "step": 150
+    },
+    {
+      "epoch": 0.1389491966999566,
+      "grad_norm": 0.9205271005630493,
+      "learning_rate": 0.0002,
+      "loss": 0.6187,
+      "step": 160
+    },
+    {
+      "epoch": 0.14763352149370387,
+      "grad_norm": 0.6115465760231018,
+      "learning_rate": 0.0002,
+      "loss": 0.6198,
+      "step": 170
+    },
+    {
+      "epoch": 0.15631784628745116,
+      "grad_norm": 0.6563206315040588,
+      "learning_rate": 0.0002,
+      "loss": 0.6065,
+      "step": 180
+    },
+    {
+      "epoch": 0.16500217108119844,
+      "grad_norm": 1.2801162004470825,
+      "learning_rate": 0.0002,
+      "loss": 0.6164,
+      "step": 190
+    },
+    {
+      "epoch": 0.17368649587494572,
+      "grad_norm": 0.5789668560028076,
+      "learning_rate": 0.0002,
+      "loss": 0.5769,
+      "step": 200
+    },
+    {
+      "epoch": 0.182370820668693,
+      "grad_norm": 0.6813567876815796,
+      "learning_rate": 0.0002,
+      "loss": 0.6087,
+      "step": 210
+    },
+    {
+      "epoch": 0.1910551454624403,
+      "grad_norm": 0.5566724538803101,
+      "learning_rate": 0.0002,
+      "loss": 0.6034,
+      "step": 220
+    },
+    {
+      "epoch": 0.19973947025618757,
+      "grad_norm": 0.7519480586051941,
+      "learning_rate": 0.0002,
+      "loss": 0.5839,
+      "step": 230
+    },
+    {
+      "epoch": 0.20842379504993486,
+      "grad_norm": 0.6408759951591492,
+      "learning_rate": 0.0002,
+      "loss": 0.5992,
+      "step": 240
+    },
+    {
+      "epoch": 0.21710811984368214,
+      "grad_norm": 0.5549838542938232,
+      "learning_rate": 0.0002,
+      "loss": 0.5965,
+      "step": 250
+    },
+    {
+      "epoch": 0.22579244463742945,
+      "grad_norm": 0.5199561715126038,
+      "learning_rate": 0.0002,
+      "loss": 0.5733,
+      "step": 260
+    },
+    {
+      "epoch": 0.23447676943117673,
+      "grad_norm": 0.6727754473686218,
+      "learning_rate": 0.0002,
+      "loss": 0.5886,
+      "step": 270
+    },
+    {
+      "epoch": 0.24316109422492402,
+      "grad_norm": 0.6133880019187927,
+      "learning_rate": 0.0002,
+      "loss": 0.5917,
+      "step": 280
+    },
+    {
+      "epoch": 0.2518454190186713,
+      "grad_norm": 0.5724732279777527,
+      "learning_rate": 0.0002,
+      "loss": 0.5565,
+      "step": 290
+    },
+    {
+      "epoch": 0.26052974381241856,
+      "grad_norm": 0.5647387504577637,
+      "learning_rate": 0.0002,
+      "loss": 0.5755,
+      "step": 300
+    },
+    {
+      "epoch": 0.26921406860616587,
+      "grad_norm": 0.5221774578094482,
+      "learning_rate": 0.0002,
+      "loss": 0.5629,
+      "step": 310
+    },
+    {
+      "epoch": 0.2778983933999132,
+      "grad_norm": 0.6939321160316467,
+      "learning_rate": 0.0002,
+      "loss": 0.5807,
+      "step": 320
+    },
+    {
+      "epoch": 0.28658271819366044,
+      "grad_norm": 0.6391360759735107,
+      "learning_rate": 0.0002,
+      "loss": 0.5584,
+      "step": 330
+    },
+    {
+      "epoch": 0.29526704298740775,
+      "grad_norm": 0.6072664260864258,
+      "learning_rate": 0.0002,
+      "loss": 0.5979,
+      "step": 340
+    },
+    {
+      "epoch": 0.303951367781155,
+      "grad_norm": 0.5596950650215149,
+      "learning_rate": 0.0002,
+      "loss": 0.5448,
+      "step": 350
+    },
+    {
+      "epoch": 0.3126356925749023,
+      "grad_norm": 0.5198812484741211,
+      "learning_rate": 0.0002,
+      "loss": 0.5436,
+      "step": 360
+    },
+    {
+      "epoch": 0.32132001736864957,
+      "grad_norm": 0.5412276387214661,
+      "learning_rate": 0.0002,
+      "loss": 0.5524,
+      "step": 370
+    },
+    {
+      "epoch": 0.3300043421623969,
+      "grad_norm": 0.6245805621147156,
+      "learning_rate": 0.0002,
+      "loss": 0.5465,
+      "step": 380
+    },
+    {
+      "epoch": 0.33868866695614414,
+      "grad_norm": 0.5097344517707825,
+      "learning_rate": 0.0002,
+      "loss": 0.5522,
+      "step": 390
+    },
+    {
+      "epoch": 0.34737299174989145,
+      "grad_norm": 0.6196363568305969,
+      "learning_rate": 0.0002,
+      "loss": 0.5478,
+      "step": 400
+    },
+    {
+      "epoch": 0.35605731654363876,
+      "grad_norm": 0.5543810129165649,
+      "learning_rate": 0.0002,
+      "loss": 0.5566,
+      "step": 410
+    },
+    {
+      "epoch": 0.364741641337386,
+      "grad_norm": 0.5921859741210938,
+      "learning_rate": 0.0002,
+      "loss": 0.5583,
+      "step": 420
+    },
+    {
+      "epoch": 0.3734259661311333,
+      "grad_norm": 0.5664304494857788,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 430
+    },
+    {
+      "epoch": 0.3821102909248806,
+      "grad_norm": 0.5977340340614319,
+      "learning_rate": 0.0002,
+      "loss": 0.5438,
+      "step": 440
+    },
+    {
+      "epoch": 0.3907946157186279,
+      "grad_norm": 0.6227905750274658,
+      "learning_rate": 0.0002,
+      "loss": 0.5419,
+      "step": 450
+    },
+    {
+      "epoch": 0.39947894051237515,
+      "grad_norm": 0.6239493489265442,
+      "learning_rate": 0.0002,
+      "loss": 0.5055,
+      "step": 460
+    },
+    {
+      "epoch": 0.40816326530612246,
+      "grad_norm": 0.7173473238945007,
+      "learning_rate": 0.0002,
+      "loss": 0.5436,
+      "step": 470
+    },
+    {
+      "epoch": 0.4168475900998697,
+      "grad_norm": 0.581386387348175,
+      "learning_rate": 0.0002,
+      "loss": 0.5413,
+      "step": 480
+    },
+    {
+      "epoch": 0.425531914893617,
+      "grad_norm": 0.8495563864707947,
+      "learning_rate": 0.0002,
+      "loss": 0.5313,
+      "step": 490
+    },
+    {
+      "epoch": 0.4342162396873643,
+      "grad_norm": 0.7008602023124695,
+      "learning_rate": 0.0002,
+      "loss": 0.5408,
+      "step": 500
+    },
+    {
+      "epoch": 0.4429005644811116,
+      "grad_norm": 0.7453230023384094,
+      "learning_rate": 0.0002,
+      "loss": 0.5174,
+      "step": 510
+    },
+    {
+      "epoch": 0.4515848892748589,
+      "grad_norm": 0.7419916987419128,
+      "learning_rate": 0.0002,
+      "loss": 0.5358,
+      "step": 520
+    },
+    {
+      "epoch": 0.46026921406860616,
+      "grad_norm": 0.5828579664230347,
+      "learning_rate": 0.0002,
+      "loss": 0.5382,
+      "step": 530
+    },
+    {
+      "epoch": 0.46895353886235347,
+      "grad_norm": 0.7366796135902405,
+      "learning_rate": 0.0002,
+      "loss": 0.5182,
+      "step": 540
+    },
+    {
+      "epoch": 0.4776378636561007,
+      "grad_norm": 0.5648167729377747,
+      "learning_rate": 0.0002,
+      "loss": 0.5158,
+      "step": 550
+    },
+    {
+      "epoch": 0.48632218844984804,
+      "grad_norm": 0.6356498003005981,
+      "learning_rate": 0.0002,
+      "loss": 0.5097,
+      "step": 560
+    },
+    {
+      "epoch": 0.4950065132435953,
+      "grad_norm": 0.6379111409187317,
+      "learning_rate": 0.0002,
+      "loss": 0.5235,
+      "step": 570
+    },
+    {
+      "epoch": 0.5036908380373426,
+      "grad_norm": 0.6728289723396301,
+      "learning_rate": 0.0002,
+      "loss": 0.4877,
+      "step": 580
+    },
+    {
+      "epoch": 0.5123751628310899,
+      "grad_norm": 0.6453984379768372,
+      "learning_rate": 0.0002,
+      "loss": 0.4977,
+      "step": 590
+    },
+    {
+      "epoch": 0.5210594876248371,
+      "grad_norm": 0.7061713933944702,
+      "learning_rate": 0.0002,
+      "loss": 0.5135,
+      "step": 600
+    },
+    {
+      "epoch": 0.5297438124185845,
+      "grad_norm": 0.8628040552139282,
+      "learning_rate": 0.0002,
+      "loss": 0.4973,
+      "step": 610
+    },
+    {
+      "epoch": 0.5384281372123317,
+      "grad_norm": 1.0208714008331299,
+      "learning_rate": 0.0002,
+      "loss": 0.5069,
+      "step": 620
+    },
+    {
+      "epoch": 0.547112462006079,
+      "grad_norm": 0.8920316100120544,
+      "learning_rate": 0.0002,
+      "loss": 0.4906,
+      "step": 630
+    },
+    {
+      "epoch": 0.5557967867998264,
+      "grad_norm": 0.7505720853805542,
+      "learning_rate": 0.0002,
+      "loss": 0.5252,
+      "step": 640
+    },
+    {
+      "epoch": 0.5644811115935736,
+      "grad_norm": 0.7085066437721252,
+      "learning_rate": 0.0002,
+      "loss": 0.4916,
+      "step": 650
+    },
+    {
+      "epoch": 0.5731654363873209,
+      "grad_norm": 0.9441248774528503,
+      "learning_rate": 0.0002,
+      "loss": 0.5074,
+      "step": 660
+    },
+    {
+      "epoch": 0.5818497611810681,
+      "grad_norm": 0.5366020202636719,
+      "learning_rate": 0.0002,
+      "loss": 0.4915,
+      "step": 670
+    },
+    {
+      "epoch": 0.5905340859748155,
+      "grad_norm": 0.5895682573318481,
+      "learning_rate": 0.0002,
+      "loss": 0.4824,
+      "step": 680
+    },
+    {
+      "epoch": 0.5992184107685627,
+      "grad_norm": 0.9745811820030212,
+      "learning_rate": 0.0002,
+      "loss": 0.4954,
+      "step": 690
+    },
+    {
+      "epoch": 0.60790273556231,
+      "grad_norm": 0.7926701903343201,
+      "learning_rate": 0.0002,
+      "loss": 0.4916,
+      "step": 700
+    },
+    {
+      "epoch": 0.6165870603560574,
+      "grad_norm": 0.5723416209220886,
+      "learning_rate": 0.0002,
+      "loss": 0.4691,
+      "step": 710
+    },
+    {
+      "epoch": 0.6252713851498046,
+      "grad_norm": 0.6463170051574707,
+      "learning_rate": 0.0002,
+      "loss": 0.4941,
+      "step": 720
+    },
+    {
+      "epoch": 0.6339557099435519,
+      "grad_norm": 0.6526642441749573,
+      "learning_rate": 0.0002,
+      "loss": 0.4702,
+      "step": 730
+    },
+    {
+      "epoch": 0.6426400347372991,
+      "grad_norm": 0.7835870385169983,
+      "learning_rate": 0.0002,
+      "loss": 0.4646,
+      "step": 740
+    },
+    {
+      "epoch": 0.6513243595310465,
+      "grad_norm": 0.7449156641960144,
+      "learning_rate": 0.0002,
+      "loss": 0.456,
+      "step": 750
+    },
+    {
+      "epoch": 0.6600086843247938,
+      "grad_norm": 0.6138056516647339,
+      "learning_rate": 0.0002,
+      "loss": 0.4664,
+      "step": 760
+    },
+    {
+      "epoch": 0.668693009118541,
+      "grad_norm": 0.6755684614181519,
+      "learning_rate": 0.0002,
+      "loss": 0.4743,
+      "step": 770
+    },
+    {
+      "epoch": 0.6773773339122883,
+      "grad_norm": 0.7654828429222107,
+      "learning_rate": 0.0002,
+      "loss": 0.4624,
+      "step": 780
+    },
+    {
+      "epoch": 0.6860616587060356,
+      "grad_norm": 0.8211193084716797,
+      "learning_rate": 0.0002,
+      "loss": 0.482,
+      "step": 790
+    },
+    {
+      "epoch": 0.6947459834997829,
+      "grad_norm": 0.6197503805160522,
+      "learning_rate": 0.0002,
+      "loss": 0.4783,
+      "step": 800
+    },
+    {
+      "epoch": 0.7034303082935301,
+      "grad_norm": 0.8030904531478882,
+      "learning_rate": 0.0002,
+      "loss": 0.461,
+      "step": 810
+    },
+    {
+      "epoch": 0.7121146330872775,
+      "grad_norm": 0.5908632278442383,
+      "learning_rate": 0.0002,
+      "loss": 0.4726,
+      "step": 820
+    },
+    {
+      "epoch": 0.7207989578810248,
+      "grad_norm": 0.867847204208374,
+      "learning_rate": 0.0002,
+      "loss": 0.4683,
+      "step": 830
+    },
+    {
+      "epoch": 0.729483282674772,
+      "grad_norm": 0.8335438966751099,
+      "learning_rate": 0.0002,
+      "loss": 0.4494,
+      "step": 840
+    },
+    {
+      "epoch": 0.7381676074685193,
+      "grad_norm": 0.7276930212974548,
+      "learning_rate": 0.0002,
+      "loss": 0.4446,
+      "step": 850
+    },
+    {
+      "epoch": 0.7468519322622666,
+      "grad_norm": 0.6241617202758789,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 860
+    },
+    {
+      "epoch": 0.7555362570560139,
+      "grad_norm": 0.6197535395622253,
+      "learning_rate": 0.0002,
+      "loss": 0.4652,
+      "step": 870
+    },
+    {
+      "epoch": 0.7642205818497612,
+      "grad_norm": 0.7276974320411682,
+      "learning_rate": 0.0002,
+      "loss": 0.4518,
+      "step": 880
+    },
+    {
+      "epoch": 0.7729049066435084,
+      "grad_norm": 0.7494757771492004,
+      "learning_rate": 0.0002,
+      "loss": 0.4533,
+      "step": 890
+    },
+    {
+      "epoch": 0.7815892314372558,
+      "grad_norm": 0.6311206817626953,
+      "learning_rate": 0.0002,
+      "loss": 0.4534,
+      "step": 900
+    },
+    {
+      "epoch": 0.790273556231003,
+      "grad_norm": 0.6753235459327698,
+      "learning_rate": 0.0002,
+      "loss": 0.4435,
+      "step": 910
+    },
+    {
+      "epoch": 0.7989578810247503,
+      "grad_norm": 0.7800608277320862,
+      "learning_rate": 0.0002,
+      "loss": 0.4602,
+      "step": 920
+    },
+    {
+      "epoch": 0.8076422058184977,
+      "grad_norm": 0.6053752899169922,
+      "learning_rate": 0.0002,
+      "loss": 0.4502,
+      "step": 930
+    },
+    {
+      "epoch": 0.8163265306122449,
+      "grad_norm": 0.8059675097465515,
+      "learning_rate": 0.0002,
+      "loss": 0.4454,
+      "step": 940
+    },
+    {
+      "epoch": 0.8250108554059922,
+      "grad_norm": 0.7138864398002625,
+      "learning_rate": 0.0002,
+      "loss": 0.4626,
+      "step": 950
+    },
+    {
+      "epoch": 0.8336951801997394,
+      "grad_norm": 0.7769238948822021,
+      "learning_rate": 0.0002,
+      "loss": 0.4327,
+      "step": 960
+    },
+    {
+      "epoch": 0.8423795049934868,
+      "grad_norm": 0.7062289118766785,
+      "learning_rate": 0.0002,
+      "loss": 0.4478,
+      "step": 970
+    },
+    {
+      "epoch": 0.851063829787234,
+      "grad_norm": 0.744469940662384,
+      "learning_rate": 0.0002,
+      "loss": 0.4329,
+      "step": 980
+    },
+    {
+      "epoch": 0.8597481545809813,
+      "grad_norm": 0.6722288727760315,
+      "learning_rate": 0.0002,
+      "loss": 0.4303,
+      "step": 990
+    },
+    {
+      "epoch": 0.8684324793747286,
+      "grad_norm": 0.7886863946914673,
+      "learning_rate": 0.0002,
+      "loss": 0.4232,
+      "step": 1000
+    },
+    {
+      "epoch": 0.8771168041684759,
+      "grad_norm": 0.8501645922660828,
+      "learning_rate": 0.0002,
+      "loss": 0.4201,
+      "step": 1010
+    },
+    {
+      "epoch": 0.8858011289622232,
+      "grad_norm": 0.9484928250312805,
+      "learning_rate": 0.0002,
+      "loss": 0.4491,
+      "step": 1020
+    },
+    {
+      "epoch": 0.8944854537559704,
+      "grad_norm": 0.6376281976699829,
+      "learning_rate": 0.0002,
+      "loss": 0.4121,
+      "step": 1030
+    },
+    {
+      "epoch": 0.9031697785497178,
+      "grad_norm": 0.5941802263259888,
+      "learning_rate": 0.0002,
+      "loss": 0.4223,
+      "step": 1040
+    },
+    {
+      "epoch": 0.9118541033434651,
+      "grad_norm": 0.75499027967453,
+      "learning_rate": 0.0002,
+      "loss": 0.4319,
+      "step": 1050
+    },
+    {
+      "epoch": 0.9205384281372123,
+      "grad_norm": 0.8150404095649719,
+      "learning_rate": 0.0002,
+      "loss": 0.4286,
+      "step": 1060
+    },
+    {
+      "epoch": 0.9292227529309596,
+      "grad_norm": 0.8157607316970825,
+      "learning_rate": 0.0002,
+      "loss": 0.4379,
+      "step": 1070
+    },
+    {
+      "epoch": 0.9379070777247069,
+      "grad_norm": 0.6417438387870789,
+      "learning_rate": 0.0002,
+      "loss": 0.4459,
+      "step": 1080
+    },
+    {
+      "epoch": 0.9465914025184542,
+      "grad_norm": 0.8608890175819397,
+      "learning_rate": 0.0002,
+      "loss": 0.4161,
+      "step": 1090
+    },
+    {
+      "epoch": 0.9552757273122015,
+      "grad_norm": 0.5620037913322449,
+      "learning_rate": 0.0002,
+      "loss": 0.4155,
+      "step": 1100
+    },
+    {
+      "epoch": 0.9639600521059488,
+      "grad_norm": 1.3118562698364258,
+      "learning_rate": 0.0002,
+      "loss": 0.4215,
+      "step": 1110
+    },
+    {
+      "epoch": 0.9726443768996961,
+      "grad_norm": 0.8373771905899048,
+      "learning_rate": 0.0002,
+      "loss": 0.4017,
+      "step": 1120
+    },
+    {
+      "epoch": 0.9813287016934433,
+      "grad_norm": 0.7725268602371216,
+      "learning_rate": 0.0002,
+      "loss": 0.4014,
+      "step": 1130
+    },
+    {
+      "epoch": 0.9900130264871906,
+      "grad_norm": 0.7564753890037537,
+      "learning_rate": 0.0002,
+      "loss": 0.4143,
+      "step": 1140
+    },
+    {
+      "epoch": 0.998697351280938,
+      "grad_norm": 0.7307513952255249,
+      "learning_rate": 0.0002,
+      "loss": 0.4085,
+      "step": 1150
+    },
+    {
+      "epoch": 0.9995657837603127,
+      "eval_loss": 0.4137004315853119,
+      "eval_runtime": 51.9598,
+      "eval_samples_per_second": 7.025,
+      "eval_steps_per_second": 0.885,
+      "step": 1151
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 9208,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.05217478033408e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-1151/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e5ef1c767c8b99de039fadb161f732b5195c895194f4e546914308c1708a156
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ce2ca0e5a8f1724c3cd6ab2920c3043d09ff6609b0743d12d5c6ddeddbd08ca
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d95ccbc65252266d500d77524ac5a5ae076e86bff669d661cb1357f0be52b6a
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bd0e288c4e235140722e183bd42fc8b1192cdaa50afd6e3194f067aa17cfcd8
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5201e5a46753f5a6442c657ff3d69d0266209fc8a34a55732e8939158d01c5c8
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/trainer_state.json

@@ -0,0 +1,1659 @@
+{
+  "best_metric": 0.3223828971385956,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303",
+  "epoch": 2.0,
+  "eval_steps": 10,
+  "global_step": 2303,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008684324793747287,
+      "grad_norm": 1.1732063293457031,
+      "learning_rate": 0.0002,
+      "loss": 1.7067,
+      "step": 10
+    },
+    {
+      "epoch": 0.017368649587494574,
+      "grad_norm": 1.7461568117141724,
+      "learning_rate": 0.0002,
+      "loss": 1.1596,
+      "step": 20
+    },
+    {
+      "epoch": 0.026052974381241857,
+      "grad_norm": 1.1278692483901978,
+      "learning_rate": 0.0002,
+      "loss": 0.9883,
+      "step": 30
+    },
+    {
+      "epoch": 0.03473729917498915,
+      "grad_norm": 1.2258220911026,
+      "learning_rate": 0.0002,
+      "loss": 0.9004,
+      "step": 40
+    },
+    {
+      "epoch": 0.04342162396873643,
+      "grad_norm": 1.0578727722167969,
+      "learning_rate": 0.0002,
+      "loss": 0.7675,
+      "step": 50
+    },
+    {
+      "epoch": 0.052105948762483714,
+      "grad_norm": 1.060376763343811,
+      "learning_rate": 0.0002,
+      "loss": 0.7039,
+      "step": 60
+    },
+    {
+      "epoch": 0.060790273556231005,
+      "grad_norm": 0.9346579313278198,
+      "learning_rate": 0.0002,
+      "loss": 0.7107,
+      "step": 70
+    },
+    {
+      "epoch": 0.0694745983499783,
+      "grad_norm": 0.7925026416778564,
+      "learning_rate": 0.0002,
+      "loss": 0.6864,
+      "step": 80
+    },
+    {
+      "epoch": 0.07815892314372558,
+      "grad_norm": 0.7674112319946289,
+      "learning_rate": 0.0002,
+      "loss": 0.6753,
+      "step": 90
+    },
+    {
+      "epoch": 0.08684324793747286,
+      "grad_norm": 0.8018007278442383,
+      "learning_rate": 0.0002,
+      "loss": 0.628,
+      "step": 100
+    },
+    {
+      "epoch": 0.09552757273122015,
+      "grad_norm": 0.8229703903198242,
+      "learning_rate": 0.0002,
+      "loss": 0.6552,
+      "step": 110
+    },
+    {
+      "epoch": 0.10421189752496743,
+      "grad_norm": 1.0095752477645874,
+      "learning_rate": 0.0002,
+      "loss": 0.6256,
+      "step": 120
+    },
+    {
+      "epoch": 0.11289622231871473,
+      "grad_norm": 1.26262629032135,
+      "learning_rate": 0.0002,
+      "loss": 0.6208,
+      "step": 130
+    },
+    {
+      "epoch": 0.12158054711246201,
+      "grad_norm": 0.6695842742919922,
+      "learning_rate": 0.0002,
+      "loss": 0.6167,
+      "step": 140
+    },
+    {
+      "epoch": 0.13026487190620928,
+      "grad_norm": 0.8266357779502869,
+      "learning_rate": 0.0002,
+      "loss": 0.6326,
+      "step": 150
+    },
+    {
+      "epoch": 0.1389491966999566,
+      "grad_norm": 0.9205271005630493,
+      "learning_rate": 0.0002,
+      "loss": 0.6187,
+      "step": 160
+    },
+    {
+      "epoch": 0.14763352149370387,
+      "grad_norm": 0.6115465760231018,
+      "learning_rate": 0.0002,
+      "loss": 0.6198,
+      "step": 170
+    },
+    {
+      "epoch": 0.15631784628745116,
+      "grad_norm": 0.6563206315040588,
+      "learning_rate": 0.0002,
+      "loss": 0.6065,
+      "step": 180
+    },
+    {
+      "epoch": 0.16500217108119844,
+      "grad_norm": 1.2801162004470825,
+      "learning_rate": 0.0002,
+      "loss": 0.6164,
+      "step": 190
+    },
+    {
+      "epoch": 0.17368649587494572,
+      "grad_norm": 0.5789668560028076,
+      "learning_rate": 0.0002,
+      "loss": 0.5769,
+      "step": 200
+    },
+    {
+      "epoch": 0.182370820668693,
+      "grad_norm": 0.6813567876815796,
+      "learning_rate": 0.0002,
+      "loss": 0.6087,
+      "step": 210
+    },
+    {
+      "epoch": 0.1910551454624403,
+      "grad_norm": 0.5566724538803101,
+      "learning_rate": 0.0002,
+      "loss": 0.6034,
+      "step": 220
+    },
+    {
+      "epoch": 0.19973947025618757,
+      "grad_norm": 0.7519480586051941,
+      "learning_rate": 0.0002,
+      "loss": 0.5839,
+      "step": 230
+    },
+    {
+      "epoch": 0.20842379504993486,
+      "grad_norm": 0.6408759951591492,
+      "learning_rate": 0.0002,
+      "loss": 0.5992,
+      "step": 240
+    },
+    {
+      "epoch": 0.21710811984368214,
+      "grad_norm": 0.5549838542938232,
+      "learning_rate": 0.0002,
+      "loss": 0.5965,
+      "step": 250
+    },
+    {
+      "epoch": 0.22579244463742945,
+      "grad_norm": 0.5199561715126038,
+      "learning_rate": 0.0002,
+      "loss": 0.5733,
+      "step": 260
+    },
+    {
+      "epoch": 0.23447676943117673,
+      "grad_norm": 0.6727754473686218,
+      "learning_rate": 0.0002,
+      "loss": 0.5886,
+      "step": 270
+    },
+    {
+      "epoch": 0.24316109422492402,
+      "grad_norm": 0.6133880019187927,
+      "learning_rate": 0.0002,
+      "loss": 0.5917,
+      "step": 280
+    },
+    {
+      "epoch": 0.2518454190186713,
+      "grad_norm": 0.5724732279777527,
+      "learning_rate": 0.0002,
+      "loss": 0.5565,
+      "step": 290
+    },
+    {
+      "epoch": 0.26052974381241856,
+      "grad_norm": 0.5647387504577637,
+      "learning_rate": 0.0002,
+      "loss": 0.5755,
+      "step": 300
+    },
+    {
+      "epoch": 0.26921406860616587,
+      "grad_norm": 0.5221774578094482,
+      "learning_rate": 0.0002,
+      "loss": 0.5629,
+      "step": 310
+    },
+    {
+      "epoch": 0.2778983933999132,
+      "grad_norm": 0.6939321160316467,
+      "learning_rate": 0.0002,
+      "loss": 0.5807,
+      "step": 320
+    },
+    {
+      "epoch": 0.28658271819366044,
+      "grad_norm": 0.6391360759735107,
+      "learning_rate": 0.0002,
+      "loss": 0.5584,
+      "step": 330
+    },
+    {
+      "epoch": 0.29526704298740775,
+      "grad_norm": 0.6072664260864258,
+      "learning_rate": 0.0002,
+      "loss": 0.5979,
+      "step": 340
+    },
+    {
+      "epoch": 0.303951367781155,
+      "grad_norm": 0.5596950650215149,
+      "learning_rate": 0.0002,
+      "loss": 0.5448,
+      "step": 350
+    },
+    {
+      "epoch": 0.3126356925749023,
+      "grad_norm": 0.5198812484741211,
+      "learning_rate": 0.0002,
+      "loss": 0.5436,
+      "step": 360
+    },
+    {
+      "epoch": 0.32132001736864957,
+      "grad_norm": 0.5412276387214661,
+      "learning_rate": 0.0002,
+      "loss": 0.5524,
+      "step": 370
+    },
+    {
+      "epoch": 0.3300043421623969,
+      "grad_norm": 0.6245805621147156,
+      "learning_rate": 0.0002,
+      "loss": 0.5465,
+      "step": 380
+    },
+    {
+      "epoch": 0.33868866695614414,
+      "grad_norm": 0.5097344517707825,
+      "learning_rate": 0.0002,
+      "loss": 0.5522,
+      "step": 390
+    },
+    {
+      "epoch": 0.34737299174989145,
+      "grad_norm": 0.6196363568305969,
+      "learning_rate": 0.0002,
+      "loss": 0.5478,
+      "step": 400
+    },
+    {
+      "epoch": 0.35605731654363876,
+      "grad_norm": 0.5543810129165649,
+      "learning_rate": 0.0002,
+      "loss": 0.5566,
+      "step": 410
+    },
+    {
+      "epoch": 0.364741641337386,
+      "grad_norm": 0.5921859741210938,
+      "learning_rate": 0.0002,
+      "loss": 0.5583,
+      "step": 420
+    },
+    {
+      "epoch": 0.3734259661311333,
+      "grad_norm": 0.5664304494857788,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 430
+    },
+    {
+      "epoch": 0.3821102909248806,
+      "grad_norm": 0.5977340340614319,
+      "learning_rate": 0.0002,
+      "loss": 0.5438,
+      "step": 440
+    },
+    {
+      "epoch": 0.3907946157186279,
+      "grad_norm": 0.6227905750274658,
+      "learning_rate": 0.0002,
+      "loss": 0.5419,
+      "step": 450
+    },
+    {
+      "epoch": 0.39947894051237515,
+      "grad_norm": 0.6239493489265442,
+      "learning_rate": 0.0002,
+      "loss": 0.5055,
+      "step": 460
+    },
+    {
+      "epoch": 0.40816326530612246,
+      "grad_norm": 0.7173473238945007,
+      "learning_rate": 0.0002,
+      "loss": 0.5436,
+      "step": 470
+    },
+    {
+      "epoch": 0.4168475900998697,
+      "grad_norm": 0.581386387348175,
+      "learning_rate": 0.0002,
+      "loss": 0.5413,
+      "step": 480
+    },
+    {
+      "epoch": 0.425531914893617,
+      "grad_norm": 0.8495563864707947,
+      "learning_rate": 0.0002,
+      "loss": 0.5313,
+      "step": 490
+    },
+    {
+      "epoch": 0.4342162396873643,
+      "grad_norm": 0.7008602023124695,
+      "learning_rate": 0.0002,
+      "loss": 0.5408,
+      "step": 500
+    },
+    {
+      "epoch": 0.4429005644811116,
+      "grad_norm": 0.7453230023384094,
+      "learning_rate": 0.0002,
+      "loss": 0.5174,
+      "step": 510
+    },
+    {
+      "epoch": 0.4515848892748589,
+      "grad_norm": 0.7419916987419128,
+      "learning_rate": 0.0002,
+      "loss": 0.5358,
+      "step": 520
+    },
+    {
+      "epoch": 0.46026921406860616,
+      "grad_norm": 0.5828579664230347,
+      "learning_rate": 0.0002,
+      "loss": 0.5382,
+      "step": 530
+    },
+    {
+      "epoch": 0.46895353886235347,
+      "grad_norm": 0.7366796135902405,
+      "learning_rate": 0.0002,
+      "loss": 0.5182,
+      "step": 540
+    },
+    {
+      "epoch": 0.4776378636561007,
+      "grad_norm": 0.5648167729377747,
+      "learning_rate": 0.0002,
+      "loss": 0.5158,
+      "step": 550
+    },
+    {
+      "epoch": 0.48632218844984804,
+      "grad_norm": 0.6356498003005981,
+      "learning_rate": 0.0002,
+      "loss": 0.5097,
+      "step": 560
+    },
+    {
+      "epoch": 0.4950065132435953,
+      "grad_norm": 0.6379111409187317,
+      "learning_rate": 0.0002,
+      "loss": 0.5235,
+      "step": 570
+    },
+    {
+      "epoch": 0.5036908380373426,
+      "grad_norm": 0.6728289723396301,
+      "learning_rate": 0.0002,
+      "loss": 0.4877,
+      "step": 580
+    },
+    {
+      "epoch": 0.5123751628310899,
+      "grad_norm": 0.6453984379768372,
+      "learning_rate": 0.0002,
+      "loss": 0.4977,
+      "step": 590
+    },
+    {
+      "epoch": 0.5210594876248371,
+      "grad_norm": 0.7061713933944702,
+      "learning_rate": 0.0002,
+      "loss": 0.5135,
+      "step": 600
+    },
+    {
+      "epoch": 0.5297438124185845,
+      "grad_norm": 0.8628040552139282,
+      "learning_rate": 0.0002,
+      "loss": 0.4973,
+      "step": 610
+    },
+    {
+      "epoch": 0.5384281372123317,
+      "grad_norm": 1.0208714008331299,
+      "learning_rate": 0.0002,
+      "loss": 0.5069,
+      "step": 620
+    },
+    {
+      "epoch": 0.547112462006079,
+      "grad_norm": 0.8920316100120544,
+      "learning_rate": 0.0002,
+      "loss": 0.4906,
+      "step": 630
+    },
+    {
+      "epoch": 0.5557967867998264,
+      "grad_norm": 0.7505720853805542,
+      "learning_rate": 0.0002,
+      "loss": 0.5252,
+      "step": 640
+    },
+    {
+      "epoch": 0.5644811115935736,
+      "grad_norm": 0.7085066437721252,
+      "learning_rate": 0.0002,
+      "loss": 0.4916,
+      "step": 650
+    },
+    {
+      "epoch": 0.5731654363873209,
+      "grad_norm": 0.9441248774528503,
+      "learning_rate": 0.0002,
+      "loss": 0.5074,
+      "step": 660
+    },
+    {
+      "epoch": 0.5818497611810681,
+      "grad_norm": 0.5366020202636719,
+      "learning_rate": 0.0002,
+      "loss": 0.4915,
+      "step": 670
+    },
+    {
+      "epoch": 0.5905340859748155,
+      "grad_norm": 0.5895682573318481,
+      "learning_rate": 0.0002,
+      "loss": 0.4824,
+      "step": 680
+    },
+    {
+      "epoch": 0.5992184107685627,
+      "grad_norm": 0.9745811820030212,
+      "learning_rate": 0.0002,
+      "loss": 0.4954,
+      "step": 690
+    },
+    {
+      "epoch": 0.60790273556231,
+      "grad_norm": 0.7926701903343201,
+      "learning_rate": 0.0002,
+      "loss": 0.4916,
+      "step": 700
+    },
+    {
+      "epoch": 0.6165870603560574,
+      "grad_norm": 0.5723416209220886,
+      "learning_rate": 0.0002,
+      "loss": 0.4691,
+      "step": 710
+    },
+    {
+      "epoch": 0.6252713851498046,
+      "grad_norm": 0.6463170051574707,
+      "learning_rate": 0.0002,
+      "loss": 0.4941,
+      "step": 720
+    },
+    {
+      "epoch": 0.6339557099435519,
+      "grad_norm": 0.6526642441749573,
+      "learning_rate": 0.0002,
+      "loss": 0.4702,
+      "step": 730
+    },
+    {
+      "epoch": 0.6426400347372991,
+      "grad_norm": 0.7835870385169983,
+      "learning_rate": 0.0002,
+      "loss": 0.4646,
+      "step": 740
+    },
+    {
+      "epoch": 0.6513243595310465,
+      "grad_norm": 0.7449156641960144,
+      "learning_rate": 0.0002,
+      "loss": 0.456,
+      "step": 750
+    },
+    {
+      "epoch": 0.6600086843247938,
+      "grad_norm": 0.6138056516647339,
+      "learning_rate": 0.0002,
+      "loss": 0.4664,
+      "step": 760
+    },
+    {
+      "epoch": 0.668693009118541,
+      "grad_norm": 0.6755684614181519,
+      "learning_rate": 0.0002,
+      "loss": 0.4743,
+      "step": 770
+    },
+    {
+      "epoch": 0.6773773339122883,
+      "grad_norm": 0.7654828429222107,
+      "learning_rate": 0.0002,
+      "loss": 0.4624,
+      "step": 780
+    },
+    {
+      "epoch": 0.6860616587060356,
+      "grad_norm": 0.8211193084716797,
+      "learning_rate": 0.0002,
+      "loss": 0.482,
+      "step": 790
+    },
+    {
+      "epoch": 0.6947459834997829,
+      "grad_norm": 0.6197503805160522,
+      "learning_rate": 0.0002,
+      "loss": 0.4783,
+      "step": 800
+    },
+    {
+      "epoch": 0.7034303082935301,
+      "grad_norm": 0.8030904531478882,
+      "learning_rate": 0.0002,
+      "loss": 0.461,
+      "step": 810
+    },
+    {
+      "epoch": 0.7121146330872775,
+      "grad_norm": 0.5908632278442383,
+      "learning_rate": 0.0002,
+      "loss": 0.4726,
+      "step": 820
+    },
+    {
+      "epoch": 0.7207989578810248,
+      "grad_norm": 0.867847204208374,
+      "learning_rate": 0.0002,
+      "loss": 0.4683,
+      "step": 830
+    },
+    {
+      "epoch": 0.729483282674772,
+      "grad_norm": 0.8335438966751099,
+      "learning_rate": 0.0002,
+      "loss": 0.4494,
+      "step": 840
+    },
+    {
+      "epoch": 0.7381676074685193,
+      "grad_norm": 0.7276930212974548,
+      "learning_rate": 0.0002,
+      "loss": 0.4446,
+      "step": 850
+    },
+    {
+      "epoch": 0.7468519322622666,
+      "grad_norm": 0.6241617202758789,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 860
+    },
+    {
+      "epoch": 0.7555362570560139,
+      "grad_norm": 0.6197535395622253,
+      "learning_rate": 0.0002,
+      "loss": 0.4652,
+      "step": 870
+    },
+    {
+      "epoch": 0.7642205818497612,
+      "grad_norm": 0.7276974320411682,
+      "learning_rate": 0.0002,
+      "loss": 0.4518,
+      "step": 880
+    },
+    {
+      "epoch": 0.7729049066435084,
+      "grad_norm": 0.7494757771492004,
+      "learning_rate": 0.0002,
+      "loss": 0.4533,
+      "step": 890
+    },
+    {
+      "epoch": 0.7815892314372558,
+      "grad_norm": 0.6311206817626953,
+      "learning_rate": 0.0002,
+      "loss": 0.4534,
+      "step": 900
+    },
+    {
+      "epoch": 0.790273556231003,
+      "grad_norm": 0.6753235459327698,
+      "learning_rate": 0.0002,
+      "loss": 0.4435,
+      "step": 910
+    },
+    {
+      "epoch": 0.7989578810247503,
+      "grad_norm": 0.7800608277320862,
+      "learning_rate": 0.0002,
+      "loss": 0.4602,
+      "step": 920
+    },
+    {
+      "epoch": 0.8076422058184977,
+      "grad_norm": 0.6053752899169922,
+      "learning_rate": 0.0002,
+      "loss": 0.4502,
+      "step": 930
+    },
+    {
+      "epoch": 0.8163265306122449,
+      "grad_norm": 0.8059675097465515,
+      "learning_rate": 0.0002,
+      "loss": 0.4454,
+      "step": 940
+    },
+    {
+      "epoch": 0.8250108554059922,
+      "grad_norm": 0.7138864398002625,
+      "learning_rate": 0.0002,
+      "loss": 0.4626,
+      "step": 950
+    },
+    {
+      "epoch": 0.8336951801997394,
+      "grad_norm": 0.7769238948822021,
+      "learning_rate": 0.0002,
+      "loss": 0.4327,
+      "step": 960
+    },
+    {
+      "epoch": 0.8423795049934868,
+      "grad_norm": 0.7062289118766785,
+      "learning_rate": 0.0002,
+      "loss": 0.4478,
+      "step": 970
+    },
+    {
+      "epoch": 0.851063829787234,
+      "grad_norm": 0.744469940662384,
+      "learning_rate": 0.0002,
+      "loss": 0.4329,
+      "step": 980
+    },
+    {
+      "epoch": 0.8597481545809813,
+      "grad_norm": 0.6722288727760315,
+      "learning_rate": 0.0002,
+      "loss": 0.4303,
+      "step": 990
+    },
+    {
+      "epoch": 0.8684324793747286,
+      "grad_norm": 0.7886863946914673,
+      "learning_rate": 0.0002,
+      "loss": 0.4232,
+      "step": 1000
+    },
+    {
+      "epoch": 0.8771168041684759,
+      "grad_norm": 0.8501645922660828,
+      "learning_rate": 0.0002,
+      "loss": 0.4201,
+      "step": 1010
+    },
+    {
+      "epoch": 0.8858011289622232,
+      "grad_norm": 0.9484928250312805,
+      "learning_rate": 0.0002,
+      "loss": 0.4491,
+      "step": 1020
+    },
+    {
+      "epoch": 0.8944854537559704,
+      "grad_norm": 0.6376281976699829,
+      "learning_rate": 0.0002,
+      "loss": 0.4121,
+      "step": 1030
+    },
+    {
+      "epoch": 0.9031697785497178,
+      "grad_norm": 0.5941802263259888,
+      "learning_rate": 0.0002,
+      "loss": 0.4223,
+      "step": 1040
+    },
+    {
+      "epoch": 0.9118541033434651,
+      "grad_norm": 0.75499027967453,
+      "learning_rate": 0.0002,
+      "loss": 0.4319,
+      "step": 1050
+    },
+    {
+      "epoch": 0.9205384281372123,
+      "grad_norm": 0.8150404095649719,
+      "learning_rate": 0.0002,
+      "loss": 0.4286,
+      "step": 1060
+    },
+    {
+      "epoch": 0.9292227529309596,
+      "grad_norm": 0.8157607316970825,
+      "learning_rate": 0.0002,
+      "loss": 0.4379,
+      "step": 1070
+    },
+    {
+      "epoch": 0.9379070777247069,
+      "grad_norm": 0.6417438387870789,
+      "learning_rate": 0.0002,
+      "loss": 0.4459,
+      "step": 1080
+    },
+    {
+      "epoch": 0.9465914025184542,
+      "grad_norm": 0.8608890175819397,
+      "learning_rate": 0.0002,
+      "loss": 0.4161,
+      "step": 1090
+    },
+    {
+      "epoch": 0.9552757273122015,
+      "grad_norm": 0.5620037913322449,
+      "learning_rate": 0.0002,
+      "loss": 0.4155,
+      "step": 1100
+    },
+    {
+      "epoch": 0.9639600521059488,
+      "grad_norm": 1.3118562698364258,
+      "learning_rate": 0.0002,
+      "loss": 0.4215,
+      "step": 1110
+    },
+    {
+      "epoch": 0.9726443768996961,
+      "grad_norm": 0.8373771905899048,
+      "learning_rate": 0.0002,
+      "loss": 0.4017,
+      "step": 1120
+    },
+    {
+      "epoch": 0.9813287016934433,
+      "grad_norm": 0.7725268602371216,
+      "learning_rate": 0.0002,
+      "loss": 0.4014,
+      "step": 1130
+    },
+    {
+      "epoch": 0.9900130264871906,
+      "grad_norm": 0.7564753890037537,
+      "learning_rate": 0.0002,
+      "loss": 0.4143,
+      "step": 1140
+    },
+    {
+      "epoch": 0.998697351280938,
+      "grad_norm": 0.7307513952255249,
+      "learning_rate": 0.0002,
+      "loss": 0.4085,
+      "step": 1150
+    },
+    {
+      "epoch": 0.9995657837603127,
+      "eval_loss": 0.4137004315853119,
+      "eval_runtime": 51.9598,
+      "eval_samples_per_second": 7.025,
+      "eval_steps_per_second": 0.885,
+      "step": 1151
+    },
+    {
+      "epoch": 1.0073816760746852,
+      "grad_norm": 0.880082905292511,
+      "learning_rate": 0.0002,
+      "loss": 0.3848,
+      "step": 1160
+    },
+    {
+      "epoch": 1.0160660008684326,
+      "grad_norm": 0.7512372732162476,
+      "learning_rate": 0.0002,
+      "loss": 0.3722,
+      "step": 1170
+    },
+    {
+      "epoch": 1.0247503256621797,
+      "grad_norm": 0.6832528710365295,
+      "learning_rate": 0.0002,
+      "loss": 0.3756,
+      "step": 1180
+    },
+    {
+      "epoch": 1.033434650455927,
+      "grad_norm": 0.9188297986984253,
+      "learning_rate": 0.0002,
+      "loss": 0.3798,
+      "step": 1190
+    },
+    {
+      "epoch": 1.0421189752496742,
+      "grad_norm": 0.6144621968269348,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 1200
+    },
+    {
+      "epoch": 1.0508033000434216,
+      "grad_norm": 0.750042200088501,
+      "learning_rate": 0.0002,
+      "loss": 0.3893,
+      "step": 1210
+    },
+    {
+      "epoch": 1.059487624837169,
+      "grad_norm": 0.6084040999412537,
+      "learning_rate": 0.0002,
+      "loss": 0.3793,
+      "step": 1220
+    },
+    {
+      "epoch": 1.068171949630916,
+      "grad_norm": 0.8569493889808655,
+      "learning_rate": 0.0002,
+      "loss": 0.3857,
+      "step": 1230
+    },
+    {
+      "epoch": 1.0768562744246635,
+      "grad_norm": 0.8420213460922241,
+      "learning_rate": 0.0002,
+      "loss": 0.381,
+      "step": 1240
+    },
+    {
+      "epoch": 1.0855405992184108,
+      "grad_norm": 0.5716078281402588,
+      "learning_rate": 0.0002,
+      "loss": 0.3905,
+      "step": 1250
+    },
+    {
+      "epoch": 1.094224924012158,
+      "grad_norm": 0.6244675517082214,
+      "learning_rate": 0.0002,
+      "loss": 0.3578,
+      "step": 1260
+    },
+    {
+      "epoch": 1.1029092488059054,
+      "grad_norm": 0.7346081733703613,
+      "learning_rate": 0.0002,
+      "loss": 0.3817,
+      "step": 1270
+    },
+    {
+      "epoch": 1.1115935735996527,
+      "grad_norm": 1.0054655075073242,
+      "learning_rate": 0.0002,
+      "loss": 0.3738,
+      "step": 1280
+    },
+    {
+      "epoch": 1.1202778983933999,
+      "grad_norm": 0.7442952394485474,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 1290
+    },
+    {
+      "epoch": 1.1289622231871472,
+      "grad_norm": 0.7852597236633301,
+      "learning_rate": 0.0002,
+      "loss": 0.3996,
+      "step": 1300
+    },
+    {
+      "epoch": 1.1376465479808946,
+      "grad_norm": 0.9616802334785461,
+      "learning_rate": 0.0002,
+      "loss": 0.3878,
+      "step": 1310
+    },
+    {
+      "epoch": 1.1463308727746417,
+      "grad_norm": 0.7134007811546326,
+      "learning_rate": 0.0002,
+      "loss": 0.3747,
+      "step": 1320
+    },
+    {
+      "epoch": 1.155015197568389,
+      "grad_norm": 0.6189036965370178,
+      "learning_rate": 0.0002,
+      "loss": 0.3609,
+      "step": 1330
+    },
+    {
+      "epoch": 1.1636995223621363,
+      "grad_norm": 0.7725108861923218,
+      "learning_rate": 0.0002,
+      "loss": 0.3793,
+      "step": 1340
+    },
+    {
+      "epoch": 1.1723838471558836,
+      "grad_norm": 0.8503804802894592,
+      "learning_rate": 0.0002,
+      "loss": 0.3687,
+      "step": 1350
+    },
+    {
+      "epoch": 1.181068171949631,
+      "grad_norm": 0.7709351181983948,
+      "learning_rate": 0.0002,
+      "loss": 0.3767,
+      "step": 1360
+    },
+    {
+      "epoch": 1.1897524967433781,
+      "grad_norm": 0.9070065021514893,
+      "learning_rate": 0.0002,
+      "loss": 0.3914,
+      "step": 1370
+    },
+    {
+      "epoch": 1.1984368215371255,
+      "grad_norm": 0.7514732480049133,
+      "learning_rate": 0.0002,
+      "loss": 0.3962,
+      "step": 1380
+    },
+    {
+      "epoch": 1.2071211463308729,
+      "grad_norm": 0.9331297874450684,
+      "learning_rate": 0.0002,
+      "loss": 0.3597,
+      "step": 1390
+    },
+    {
+      "epoch": 1.21580547112462,
+      "grad_norm": 0.9618472456932068,
+      "learning_rate": 0.0002,
+      "loss": 0.3764,
+      "step": 1400
+    },
+    {
+      "epoch": 1.2244897959183674,
+      "grad_norm": 0.8956435918807983,
+      "learning_rate": 0.0002,
+      "loss": 0.3709,
+      "step": 1410
+    },
+    {
+      "epoch": 1.2331741207121145,
+      "grad_norm": 0.9150217175483704,
+      "learning_rate": 0.0002,
+      "loss": 0.3775,
+      "step": 1420
+    },
+    {
+      "epoch": 1.2418584455058619,
+      "grad_norm": 0.6232621073722839,
+      "learning_rate": 0.0002,
+      "loss": 0.3656,
+      "step": 1430
+    },
+    {
+      "epoch": 1.2505427702996093,
+      "grad_norm": 0.980364978313446,
+      "learning_rate": 0.0002,
+      "loss": 0.3831,
+      "step": 1440
+    },
+    {
+      "epoch": 1.2592270950933564,
+      "grad_norm": 0.8638973236083984,
+      "learning_rate": 0.0002,
+      "loss": 0.3514,
+      "step": 1450
+    },
+    {
+      "epoch": 1.2679114198871038,
+      "grad_norm": 0.6402583122253418,
+      "learning_rate": 0.0002,
+      "loss": 0.3659,
+      "step": 1460
+    },
+    {
+      "epoch": 1.2765957446808511,
+      "grad_norm": 0.9318592548370361,
+      "learning_rate": 0.0002,
+      "loss": 0.3772,
+      "step": 1470
+    },
+    {
+      "epoch": 1.2852800694745983,
+      "grad_norm": 0.7031506299972534,
+      "learning_rate": 0.0002,
+      "loss": 0.3727,
+      "step": 1480
+    },
+    {
+      "epoch": 1.2939643942683456,
+      "grad_norm": 0.9225306510925293,
+      "learning_rate": 0.0002,
+      "loss": 0.3615,
+      "step": 1490
+    },
+    {
+      "epoch": 1.302648719062093,
+      "grad_norm": 0.8340631127357483,
+      "learning_rate": 0.0002,
+      "loss": 0.3838,
+      "step": 1500
+    },
+    {
+      "epoch": 1.3113330438558402,
+      "grad_norm": 0.9431206583976746,
+      "learning_rate": 0.0002,
+      "loss": 0.3747,
+      "step": 1510
+    },
+    {
+      "epoch": 1.3200173686495875,
+      "grad_norm": 0.8804561495780945,
+      "learning_rate": 0.0002,
+      "loss": 0.3535,
+      "step": 1520
+    },
+    {
+      "epoch": 1.3287016934433349,
+      "grad_norm": 0.8626439571380615,
+      "learning_rate": 0.0002,
+      "loss": 0.3679,
+      "step": 1530
+    },
+    {
+      "epoch": 1.337386018237082,
+      "grad_norm": 0.8414568305015564,
+      "learning_rate": 0.0002,
+      "loss": 0.3644,
+      "step": 1540
+    },
+    {
+      "epoch": 1.3460703430308294,
+      "grad_norm": 1.127997636795044,
+      "learning_rate": 0.0002,
+      "loss": 0.3604,
+      "step": 1550
+    },
+    {
+      "epoch": 1.3547546678245768,
+      "grad_norm": 0.7714207172393799,
+      "learning_rate": 0.0002,
+      "loss": 0.3589,
+      "step": 1560
+    },
+    {
+      "epoch": 1.363438992618324,
+      "grad_norm": 0.9359990954399109,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1570
+    },
+    {
+      "epoch": 1.3721233174120713,
+      "grad_norm": 0.7754091620445251,
+      "learning_rate": 0.0002,
+      "loss": 0.3694,
+      "step": 1580
+    },
+    {
+      "epoch": 1.3808076422058184,
+      "grad_norm": 1.0162203311920166,
+      "learning_rate": 0.0002,
+      "loss": 0.36,
+      "step": 1590
+    },
+    {
+      "epoch": 1.3894919669995658,
+      "grad_norm": 0.9444917440414429,
+      "learning_rate": 0.0002,
+      "loss": 0.347,
+      "step": 1600
+    },
+    {
+      "epoch": 1.3981762917933132,
+      "grad_norm": 0.8122510313987732,
+      "learning_rate": 0.0002,
+      "loss": 0.3625,
+      "step": 1610
+    },
+    {
+      "epoch": 1.4068606165870603,
+      "grad_norm": 0.7946245670318604,
+      "learning_rate": 0.0002,
+      "loss": 0.3499,
+      "step": 1620
+    },
+    {
+      "epoch": 1.4155449413808077,
+      "grad_norm": 0.899692714214325,
+      "learning_rate": 0.0002,
+      "loss": 0.3606,
+      "step": 1630
+    },
+    {
+      "epoch": 1.4242292661745548,
+      "grad_norm": 0.652953565120697,
+      "learning_rate": 0.0002,
+      "loss": 0.3599,
+      "step": 1640
+    },
+    {
+      "epoch": 1.4329135909683022,
+      "grad_norm": 0.6942960023880005,
+      "learning_rate": 0.0002,
+      "loss": 0.3585,
+      "step": 1650
+    },
+    {
+      "epoch": 1.4415979157620495,
+      "grad_norm": 0.5737764239311218,
+      "learning_rate": 0.0002,
+      "loss": 0.3624,
+      "step": 1660
+    },
+    {
+      "epoch": 1.4502822405557967,
+      "grad_norm": 0.7966623306274414,
+      "learning_rate": 0.0002,
+      "loss": 0.3446,
+      "step": 1670
+    },
+    {
+      "epoch": 1.458966565349544,
+      "grad_norm": 0.7757281064987183,
+      "learning_rate": 0.0002,
+      "loss": 0.34,
+      "step": 1680
+    },
+    {
+      "epoch": 1.4676508901432914,
+      "grad_norm": 0.7565333843231201,
+      "learning_rate": 0.0002,
+      "loss": 0.377,
+      "step": 1690
+    },
+    {
+      "epoch": 1.4763352149370386,
+      "grad_norm": 1.2639003992080688,
+      "learning_rate": 0.0002,
+      "loss": 0.3536,
+      "step": 1700
+    },
+    {
+      "epoch": 1.485019539730786,
+      "grad_norm": 1.0366960763931274,
+      "learning_rate": 0.0002,
+      "loss": 0.3503,
+      "step": 1710
+    },
+    {
+      "epoch": 1.4937038645245333,
+      "grad_norm": 0.8083975911140442,
+      "learning_rate": 0.0002,
+      "loss": 0.3594,
+      "step": 1720
+    },
+    {
+      "epoch": 1.5023881893182804,
+      "grad_norm": 1.0527535676956177,
+      "learning_rate": 0.0002,
+      "loss": 0.369,
+      "step": 1730
+    },
+    {
+      "epoch": 1.5110725141120278,
+      "grad_norm": 0.7138310074806213,
+      "learning_rate": 0.0002,
+      "loss": 0.3424,
+      "step": 1740
+    },
+    {
+      "epoch": 1.5197568389057752,
+      "grad_norm": 0.818398118019104,
+      "learning_rate": 0.0002,
+      "loss": 0.3568,
+      "step": 1750
+    },
+    {
+      "epoch": 1.5284411636995223,
+      "grad_norm": 0.7518394589424133,
+      "learning_rate": 0.0002,
+      "loss": 0.3347,
+      "step": 1760
+    },
+    {
+      "epoch": 1.5371254884932697,
+      "grad_norm": 0.6814934611320496,
+      "learning_rate": 0.0002,
+      "loss": 0.3462,
+      "step": 1770
+    },
+    {
+      "epoch": 1.545809813287017,
+      "grad_norm": 0.49772709608078003,
+      "learning_rate": 0.0002,
+      "loss": 0.3432,
+      "step": 1780
+    },
+    {
+      "epoch": 1.5544941380807642,
+      "grad_norm": 0.7579216957092285,
+      "learning_rate": 0.0002,
+      "loss": 0.3457,
+      "step": 1790
+    },
+    {
+      "epoch": 1.5631784628745113,
+      "grad_norm": 0.7244242429733276,
+      "learning_rate": 0.0002,
+      "loss": 0.3517,
+      "step": 1800
+    },
+    {
+      "epoch": 1.571862787668259,
+      "grad_norm": 0.6358800530433655,
+      "learning_rate": 0.0002,
+      "loss": 0.3427,
+      "step": 1810
+    },
+    {
+      "epoch": 1.580547112462006,
+      "grad_norm": 0.6636850833892822,
+      "learning_rate": 0.0002,
+      "loss": 0.3482,
+      "step": 1820
+    },
+    {
+      "epoch": 1.5892314372557532,
+      "grad_norm": 0.691331148147583,
+      "learning_rate": 0.0002,
+      "loss": 0.3397,
+      "step": 1830
+    },
+    {
+      "epoch": 1.5979157620495008,
+      "grad_norm": 0.8501580357551575,
+      "learning_rate": 0.0002,
+      "loss": 0.3525,
+      "step": 1840
+    },
+    {
+      "epoch": 1.606600086843248,
+      "grad_norm": 0.8448147177696228,
+      "learning_rate": 0.0002,
+      "loss": 0.3319,
+      "step": 1850
+    },
+    {
+      "epoch": 1.615284411636995,
+      "grad_norm": 0.6357255578041077,
+      "learning_rate": 0.0002,
+      "loss": 0.3355,
+      "step": 1860
+    },
+    {
+      "epoch": 1.6239687364307425,
+      "grad_norm": 1.008238673210144,
+      "learning_rate": 0.0002,
+      "loss": 0.3592,
+      "step": 1870
+    },
+    {
+      "epoch": 1.6326530612244898,
+      "grad_norm": 0.7839747667312622,
+      "learning_rate": 0.0002,
+      "loss": 0.3342,
+      "step": 1880
+    },
+    {
+      "epoch": 1.641337386018237,
+      "grad_norm": 1.1214498281478882,
+      "learning_rate": 0.0002,
+      "loss": 0.3348,
+      "step": 1890
+    },
+    {
+      "epoch": 1.6500217108119843,
+      "grad_norm": 0.9457558989524841,
+      "learning_rate": 0.0002,
+      "loss": 0.335,
+      "step": 1900
+    },
+    {
+      "epoch": 1.6587060356057317,
+      "grad_norm": 0.8412042260169983,
+      "learning_rate": 0.0002,
+      "loss": 0.3425,
+      "step": 1910
+    },
+    {
+      "epoch": 1.6673903603994789,
+      "grad_norm": 1.0219208002090454,
+      "learning_rate": 0.0002,
+      "loss": 0.3427,
+      "step": 1920
+    },
+    {
+      "epoch": 1.6760746851932262,
+      "grad_norm": 1.079763412475586,
+      "learning_rate": 0.0002,
+      "loss": 0.3429,
+      "step": 1930
+    },
+    {
+      "epoch": 1.6847590099869736,
+      "grad_norm": 0.6026722192764282,
+      "learning_rate": 0.0002,
+      "loss": 0.3433,
+      "step": 1940
+    },
+    {
+      "epoch": 1.6934433347807207,
+      "grad_norm": 0.840270459651947,
+      "learning_rate": 0.0002,
+      "loss": 0.3438,
+      "step": 1950
+    },
+    {
+      "epoch": 1.702127659574468,
+      "grad_norm": 0.8331791758537292,
+      "learning_rate": 0.0002,
+      "loss": 0.3544,
+      "step": 1960
+    },
+    {
+      "epoch": 1.7108119843682155,
+      "grad_norm": 0.6232297420501709,
+      "learning_rate": 0.0002,
+      "loss": 0.3396,
+      "step": 1970
+    },
+    {
+      "epoch": 1.7194963091619626,
+      "grad_norm": 0.8575964570045471,
+      "learning_rate": 0.0002,
+      "loss": 0.3413,
+      "step": 1980
+    },
+    {
+      "epoch": 1.72818063395571,
+      "grad_norm": 1.4170857667922974,
+      "learning_rate": 0.0002,
+      "loss": 0.3375,
+      "step": 1990
+    },
+    {
+      "epoch": 1.7368649587494573,
+      "grad_norm": 0.7481379508972168,
+      "learning_rate": 0.0002,
+      "loss": 0.3328,
+      "step": 2000
+    },
+    {
+      "epoch": 1.7455492835432045,
+      "grad_norm": 0.6883425712585449,
+      "learning_rate": 0.0002,
+      "loss": 0.3413,
+      "step": 2010
+    },
+    {
+      "epoch": 1.7542336083369519,
+      "grad_norm": 1.1543880701065063,
+      "learning_rate": 0.0002,
+      "loss": 0.3381,
+      "step": 2020
+    },
+    {
+      "epoch": 1.7629179331306992,
+      "grad_norm": 0.8788072466850281,
+      "learning_rate": 0.0002,
+      "loss": 0.3369,
+      "step": 2030
+    },
+    {
+      "epoch": 1.7716022579244464,
+      "grad_norm": 0.7034786343574524,
+      "learning_rate": 0.0002,
+      "loss": 0.3266,
+      "step": 2040
+    },
+    {
+      "epoch": 1.7802865827181935,
+      "grad_norm": 1.041170358657837,
+      "learning_rate": 0.0002,
+      "loss": 0.3263,
+      "step": 2050
+    },
+    {
+      "epoch": 1.788970907511941,
+      "grad_norm": 0.7966195344924927,
+      "learning_rate": 0.0002,
+      "loss": 0.3309,
+      "step": 2060
+    },
+    {
+      "epoch": 1.7976552323056882,
+      "grad_norm": 0.8562559485435486,
+      "learning_rate": 0.0002,
+      "loss": 0.3174,
+      "step": 2070
+    },
+    {
+      "epoch": 1.8063395570994354,
+      "grad_norm": 0.9997917413711548,
+      "learning_rate": 0.0002,
+      "loss": 0.3384,
+      "step": 2080
+    },
+    {
+      "epoch": 1.8150238818931828,
+      "grad_norm": 0.9915640950202942,
+      "learning_rate": 0.0002,
+      "loss": 0.3291,
+      "step": 2090
+    },
+    {
+      "epoch": 1.8237082066869301,
+      "grad_norm": 0.8850535154342651,
+      "learning_rate": 0.0002,
+      "loss": 0.3266,
+      "step": 2100
+    },
+    {
+      "epoch": 1.8323925314806773,
+      "grad_norm": 0.7072052359580994,
+      "learning_rate": 0.0002,
+      "loss": 0.3269,
+      "step": 2110
+    },
+    {
+      "epoch": 1.8410768562744246,
+      "grad_norm": 0.8423166871070862,
+      "learning_rate": 0.0002,
+      "loss": 0.3302,
+      "step": 2120
+    },
+    {
+      "epoch": 1.849761181068172,
+      "grad_norm": 0.7291885614395142,
+      "learning_rate": 0.0002,
+      "loss": 0.3382,
+      "step": 2130
+    },
+    {
+      "epoch": 1.8584455058619191,
+      "grad_norm": 1.0388280153274536,
+      "learning_rate": 0.0002,
+      "loss": 0.3247,
+      "step": 2140
+    },
+    {
+      "epoch": 1.8671298306556665,
+      "grad_norm": 0.8792893886566162,
+      "learning_rate": 0.0002,
+      "loss": 0.3223,
+      "step": 2150
+    },
+    {
+      "epoch": 1.8758141554494139,
+      "grad_norm": 0.852391242980957,
+      "learning_rate": 0.0002,
+      "loss": 0.3385,
+      "step": 2160
+    },
+    {
+      "epoch": 1.884498480243161,
+      "grad_norm": 0.7536265850067139,
+      "learning_rate": 0.0002,
+      "loss": 0.3253,
+      "step": 2170
+    },
+    {
+      "epoch": 1.8931828050369084,
+      "grad_norm": 0.8531852960586548,
+      "learning_rate": 0.0002,
+      "loss": 0.3234,
+      "step": 2180
+    },
+    {
+      "epoch": 1.9018671298306558,
+      "grad_norm": 0.6752887964248657,
+      "learning_rate": 0.0002,
+      "loss": 0.3352,
+      "step": 2190
+    },
+    {
+      "epoch": 1.910551454624403,
+      "grad_norm": 0.9746645092964172,
+      "learning_rate": 0.0002,
+      "loss": 0.3344,
+      "step": 2200
+    },
+    {
+      "epoch": 1.9192357794181503,
+      "grad_norm": 0.8092345595359802,
+      "learning_rate": 0.0002,
+      "loss": 0.331,
+      "step": 2210
+    },
+    {
+      "epoch": 1.9279201042118976,
+      "grad_norm": 0.6521363854408264,
+      "learning_rate": 0.0002,
+      "loss": 0.3245,
+      "step": 2220
+    },
+    {
+      "epoch": 1.9366044290056448,
+      "grad_norm": 1.0131723880767822,
+      "learning_rate": 0.0002,
+      "loss": 0.3272,
+      "step": 2230
+    },
+    {
+      "epoch": 1.9452887537993921,
+      "grad_norm": 0.6056963205337524,
+      "learning_rate": 0.0002,
+      "loss": 0.3166,
+      "step": 2240
+    },
+    {
+      "epoch": 1.9539730785931395,
+      "grad_norm": 0.9994075894355774,
+      "learning_rate": 0.0002,
+      "loss": 0.3346,
+      "step": 2250
+    },
+    {
+      "epoch": 1.9626574033868867,
+      "grad_norm": 0.8796638250350952,
+      "learning_rate": 0.0002,
+      "loss": 0.3214,
+      "step": 2260
+    },
+    {
+      "epoch": 1.9713417281806338,
+      "grad_norm": 0.6505881547927856,
+      "learning_rate": 0.0002,
+      "loss": 0.3225,
+      "step": 2270
+    },
+    {
+      "epoch": 1.9800260529743814,
+      "grad_norm": 1.1708468198776245,
+      "learning_rate": 0.0002,
+      "loss": 0.3346,
+      "step": 2280
+    },
+    {
+      "epoch": 1.9887103777681285,
+      "grad_norm": 0.898364245891571,
+      "learning_rate": 0.0002,
+      "loss": 0.3234,
+      "step": 2290
+    },
+    {
+      "epoch": 1.9973947025618757,
+      "grad_norm": 0.7017287611961365,
+      "learning_rate": 0.0002,
+      "loss": 0.3263,
+      "step": 2300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.3223828971385956,
+      "eval_runtime": 51.2747,
+      "eval_samples_per_second": 7.119,
+      "eval_steps_per_second": 0.897,
+      "step": 2303
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 9208,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.010434956066816e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-2303/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e5ef1c767c8b99de039fadb161f732b5195c895194f4e546914308c1708a156
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80ac658e095c6b49def97efb4a301199b1183a08ae033a807792cbd2cd5ceeaa
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf9e821c34565d650ca6da3d53a48b94799b4ef57433f08c4260c9e25a8222ff
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0bab5502645fe7e45b84934beb0f080e6dc25b5cb0cd220fb29881df7ec6625
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50bfa33970050d0e3b27877e768de69b17210cac409705a83e520bd02a9ccfcf
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/trainer_state.json

@@ -0,0 +1,2472 @@
+{
+  "best_metric": 0.2924337387084961,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454",
+  "epoch": 2.9995657837603127,
+  "eval_steps": 10,
+  "global_step": 3454,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008684324793747287,
+      "grad_norm": 1.1732063293457031,
+      "learning_rate": 0.0002,
+      "loss": 1.7067,
+      "step": 10
+    },
+    {
+      "epoch": 0.017368649587494574,
+      "grad_norm": 1.7461568117141724,
+      "learning_rate": 0.0002,
+      "loss": 1.1596,
+      "step": 20
+    },
+    {
+      "epoch": 0.026052974381241857,
+      "grad_norm": 1.1278692483901978,
+      "learning_rate": 0.0002,
+      "loss": 0.9883,
+      "step": 30
+    },
+    {
+      "epoch": 0.03473729917498915,
+      "grad_norm": 1.2258220911026,
+      "learning_rate": 0.0002,
+      "loss": 0.9004,
+      "step": 40
+    },
+    {
+      "epoch": 0.04342162396873643,
+      "grad_norm": 1.0578727722167969,
+      "learning_rate": 0.0002,
+      "loss": 0.7675,
+      "step": 50
+    },
+    {
+      "epoch": 0.052105948762483714,
+      "grad_norm": 1.060376763343811,
+      "learning_rate": 0.0002,
+      "loss": 0.7039,
+      "step": 60
+    },
+    {
+      "epoch": 0.060790273556231005,
+      "grad_norm": 0.9346579313278198,
+      "learning_rate": 0.0002,
+      "loss": 0.7107,
+      "step": 70
+    },
+    {
+      "epoch": 0.0694745983499783,
+      "grad_norm": 0.7925026416778564,
+      "learning_rate": 0.0002,
+      "loss": 0.6864,
+      "step": 80
+    },
+    {
+      "epoch": 0.07815892314372558,
+      "grad_norm": 0.7674112319946289,
+      "learning_rate": 0.0002,
+      "loss": 0.6753,
+      "step": 90
+    },
+    {
+      "epoch": 0.08684324793747286,
+      "grad_norm": 0.8018007278442383,
+      "learning_rate": 0.0002,
+      "loss": 0.628,
+      "step": 100
+    },
+    {
+      "epoch": 0.09552757273122015,
+      "grad_norm": 0.8229703903198242,
+      "learning_rate": 0.0002,
+      "loss": 0.6552,
+      "step": 110
+    },
+    {
+      "epoch": 0.10421189752496743,
+      "grad_norm": 1.0095752477645874,
+      "learning_rate": 0.0002,
+      "loss": 0.6256,
+      "step": 120
+    },
+    {
+      "epoch": 0.11289622231871473,
+      "grad_norm": 1.26262629032135,
+      "learning_rate": 0.0002,
+      "loss": 0.6208,
+      "step": 130
+    },
+    {
+      "epoch": 0.12158054711246201,
+      "grad_norm": 0.6695842742919922,
+      "learning_rate": 0.0002,
+      "loss": 0.6167,
+      "step": 140
+    },
+    {
+      "epoch": 0.13026487190620928,
+      "grad_norm": 0.8266357779502869,
+      "learning_rate": 0.0002,
+      "loss": 0.6326,
+      "step": 150
+    },
+    {
+      "epoch": 0.1389491966999566,
+      "grad_norm": 0.9205271005630493,
+      "learning_rate": 0.0002,
+      "loss": 0.6187,
+      "step": 160
+    },
+    {
+      "epoch": 0.14763352149370387,
+      "grad_norm": 0.6115465760231018,
+      "learning_rate": 0.0002,
+      "loss": 0.6198,
+      "step": 170
+    },
+    {
+      "epoch": 0.15631784628745116,
+      "grad_norm": 0.6563206315040588,
+      "learning_rate": 0.0002,
+      "loss": 0.6065,
+      "step": 180
+    },
+    {
+      "epoch": 0.16500217108119844,
+      "grad_norm": 1.2801162004470825,
+      "learning_rate": 0.0002,
+      "loss": 0.6164,
+      "step": 190
+    },
+    {
+      "epoch": 0.17368649587494572,
+      "grad_norm": 0.5789668560028076,
+      "learning_rate": 0.0002,
+      "loss": 0.5769,
+      "step": 200
+    },
+    {
+      "epoch": 0.182370820668693,
+      "grad_norm": 0.6813567876815796,
+      "learning_rate": 0.0002,
+      "loss": 0.6087,
+      "step": 210
+    },
+    {
+      "epoch": 0.1910551454624403,
+      "grad_norm": 0.5566724538803101,
+      "learning_rate": 0.0002,
+      "loss": 0.6034,
+      "step": 220
+    },
+    {
+      "epoch": 0.19973947025618757,
+      "grad_norm": 0.7519480586051941,
+      "learning_rate": 0.0002,
+      "loss": 0.5839,
+      "step": 230
+    },
+    {
+      "epoch": 0.20842379504993486,
+      "grad_norm": 0.6408759951591492,
+      "learning_rate": 0.0002,
+      "loss": 0.5992,
+      "step": 240
+    },
+    {
+      "epoch": 0.21710811984368214,
+      "grad_norm": 0.5549838542938232,
+      "learning_rate": 0.0002,
+      "loss": 0.5965,
+      "step": 250
+    },
+    {
+      "epoch": 0.22579244463742945,
+      "grad_norm": 0.5199561715126038,
+      "learning_rate": 0.0002,
+      "loss": 0.5733,
+      "step": 260
+    },
+    {
+      "epoch": 0.23447676943117673,
+      "grad_norm": 0.6727754473686218,
+      "learning_rate": 0.0002,
+      "loss": 0.5886,
+      "step": 270
+    },
+    {
+      "epoch": 0.24316109422492402,
+      "grad_norm": 0.6133880019187927,
+      "learning_rate": 0.0002,
+      "loss": 0.5917,
+      "step": 280
+    },
+    {
+      "epoch": 0.2518454190186713,
+      "grad_norm": 0.5724732279777527,
+      "learning_rate": 0.0002,
+      "loss": 0.5565,
+      "step": 290
+    },
+    {
+      "epoch": 0.26052974381241856,
+      "grad_norm": 0.5647387504577637,
+      "learning_rate": 0.0002,
+      "loss": 0.5755,
+      "step": 300
+    },
+    {
+      "epoch": 0.26921406860616587,
+      "grad_norm": 0.5221774578094482,
+      "learning_rate": 0.0002,
+      "loss": 0.5629,
+      "step": 310
+    },
+    {
+      "epoch": 0.2778983933999132,
+      "grad_norm": 0.6939321160316467,
+      "learning_rate": 0.0002,
+      "loss": 0.5807,
+      "step": 320
+    },
+    {
+      "epoch": 0.28658271819366044,
+      "grad_norm": 0.6391360759735107,
+      "learning_rate": 0.0002,
+      "loss": 0.5584,
+      "step": 330
+    },
+    {
+      "epoch": 0.29526704298740775,
+      "grad_norm": 0.6072664260864258,
+      "learning_rate": 0.0002,
+      "loss": 0.5979,
+      "step": 340
+    },
+    {
+      "epoch": 0.303951367781155,
+      "grad_norm": 0.5596950650215149,
+      "learning_rate": 0.0002,
+      "loss": 0.5448,
+      "step": 350
+    },
+    {
+      "epoch": 0.3126356925749023,
+      "grad_norm": 0.5198812484741211,
+      "learning_rate": 0.0002,
+      "loss": 0.5436,
+      "step": 360
+    },
+    {
+      "epoch": 0.32132001736864957,
+      "grad_norm": 0.5412276387214661,
+      "learning_rate": 0.0002,
+      "loss": 0.5524,
+      "step": 370
+    },
+    {
+      "epoch": 0.3300043421623969,
+      "grad_norm": 0.6245805621147156,
+      "learning_rate": 0.0002,
+      "loss": 0.5465,
+      "step": 380
+    },
+    {
+      "epoch": 0.33868866695614414,
+      "grad_norm": 0.5097344517707825,
+      "learning_rate": 0.0002,
+      "loss": 0.5522,
+      "step": 390
+    },
+    {
+      "epoch": 0.34737299174989145,
+      "grad_norm": 0.6196363568305969,
+      "learning_rate": 0.0002,
+      "loss": 0.5478,
+      "step": 400
+    },
+    {
+      "epoch": 0.35605731654363876,
+      "grad_norm": 0.5543810129165649,
+      "learning_rate": 0.0002,
+      "loss": 0.5566,
+      "step": 410
+    },
+    {
+      "epoch": 0.364741641337386,
+      "grad_norm": 0.5921859741210938,
+      "learning_rate": 0.0002,
+      "loss": 0.5583,
+      "step": 420
+    },
+    {
+      "epoch": 0.3734259661311333,
+      "grad_norm": 0.5664304494857788,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 430
+    },
+    {
+      "epoch": 0.3821102909248806,
+      "grad_norm": 0.5977340340614319,
+      "learning_rate": 0.0002,
+      "loss": 0.5438,
+      "step": 440
+    },
+    {
+      "epoch": 0.3907946157186279,
+      "grad_norm": 0.6227905750274658,
+      "learning_rate": 0.0002,
+      "loss": 0.5419,
+      "step": 450
+    },
+    {
+      "epoch": 0.39947894051237515,
+      "grad_norm": 0.6239493489265442,
+      "learning_rate": 0.0002,
+      "loss": 0.5055,
+      "step": 460
+    },
+    {
+      "epoch": 0.40816326530612246,
+      "grad_norm": 0.7173473238945007,
+      "learning_rate": 0.0002,
+      "loss": 0.5436,
+      "step": 470
+    },
+    {
+      "epoch": 0.4168475900998697,
+      "grad_norm": 0.581386387348175,
+      "learning_rate": 0.0002,
+      "loss": 0.5413,
+      "step": 480
+    },
+    {
+      "epoch": 0.425531914893617,
+      "grad_norm": 0.8495563864707947,
+      "learning_rate": 0.0002,
+      "loss": 0.5313,
+      "step": 490
+    },
+    {
+      "epoch": 0.4342162396873643,
+      "grad_norm": 0.7008602023124695,
+      "learning_rate": 0.0002,
+      "loss": 0.5408,
+      "step": 500
+    },
+    {
+      "epoch": 0.4429005644811116,
+      "grad_norm": 0.7453230023384094,
+      "learning_rate": 0.0002,
+      "loss": 0.5174,
+      "step": 510
+    },
+    {
+      "epoch": 0.4515848892748589,
+      "grad_norm": 0.7419916987419128,
+      "learning_rate": 0.0002,
+      "loss": 0.5358,
+      "step": 520
+    },
+    {
+      "epoch": 0.46026921406860616,
+      "grad_norm": 0.5828579664230347,
+      "learning_rate": 0.0002,
+      "loss": 0.5382,
+      "step": 530
+    },
+    {
+      "epoch": 0.46895353886235347,
+      "grad_norm": 0.7366796135902405,
+      "learning_rate": 0.0002,
+      "loss": 0.5182,
+      "step": 540
+    },
+    {
+      "epoch": 0.4776378636561007,
+      "grad_norm": 0.5648167729377747,
+      "learning_rate": 0.0002,
+      "loss": 0.5158,
+      "step": 550
+    },
+    {
+      "epoch": 0.48632218844984804,
+      "grad_norm": 0.6356498003005981,
+      "learning_rate": 0.0002,
+      "loss": 0.5097,
+      "step": 560
+    },
+    {
+      "epoch": 0.4950065132435953,
+      "grad_norm": 0.6379111409187317,
+      "learning_rate": 0.0002,
+      "loss": 0.5235,
+      "step": 570
+    },
+    {
+      "epoch": 0.5036908380373426,
+      "grad_norm": 0.6728289723396301,
+      "learning_rate": 0.0002,
+      "loss": 0.4877,
+      "step": 580
+    },
+    {
+      "epoch": 0.5123751628310899,
+      "grad_norm": 0.6453984379768372,
+      "learning_rate": 0.0002,
+      "loss": 0.4977,
+      "step": 590
+    },
+    {
+      "epoch": 0.5210594876248371,
+      "grad_norm": 0.7061713933944702,
+      "learning_rate": 0.0002,
+      "loss": 0.5135,
+      "step": 600
+    },
+    {
+      "epoch": 0.5297438124185845,
+      "grad_norm": 0.8628040552139282,
+      "learning_rate": 0.0002,
+      "loss": 0.4973,
+      "step": 610
+    },
+    {
+      "epoch": 0.5384281372123317,
+      "grad_norm": 1.0208714008331299,
+      "learning_rate": 0.0002,
+      "loss": 0.5069,
+      "step": 620
+    },
+    {
+      "epoch": 0.547112462006079,
+      "grad_norm": 0.8920316100120544,
+      "learning_rate": 0.0002,
+      "loss": 0.4906,
+      "step": 630
+    },
+    {
+      "epoch": 0.5557967867998264,
+      "grad_norm": 0.7505720853805542,
+      "learning_rate": 0.0002,
+      "loss": 0.5252,
+      "step": 640
+    },
+    {
+      "epoch": 0.5644811115935736,
+      "grad_norm": 0.7085066437721252,
+      "learning_rate": 0.0002,
+      "loss": 0.4916,
+      "step": 650
+    },
+    {
+      "epoch": 0.5731654363873209,
+      "grad_norm": 0.9441248774528503,
+      "learning_rate": 0.0002,
+      "loss": 0.5074,
+      "step": 660
+    },
+    {
+      "epoch": 0.5818497611810681,
+      "grad_norm": 0.5366020202636719,
+      "learning_rate": 0.0002,
+      "loss": 0.4915,
+      "step": 670
+    },
+    {
+      "epoch": 0.5905340859748155,
+      "grad_norm": 0.5895682573318481,
+      "learning_rate": 0.0002,
+      "loss": 0.4824,
+      "step": 680
+    },
+    {
+      "epoch": 0.5992184107685627,
+      "grad_norm": 0.9745811820030212,
+      "learning_rate": 0.0002,
+      "loss": 0.4954,
+      "step": 690
+    },
+    {
+      "epoch": 0.60790273556231,
+      "grad_norm": 0.7926701903343201,
+      "learning_rate": 0.0002,
+      "loss": 0.4916,
+      "step": 700
+    },
+    {
+      "epoch": 0.6165870603560574,
+      "grad_norm": 0.5723416209220886,
+      "learning_rate": 0.0002,
+      "loss": 0.4691,
+      "step": 710
+    },
+    {
+      "epoch": 0.6252713851498046,
+      "grad_norm": 0.6463170051574707,
+      "learning_rate": 0.0002,
+      "loss": 0.4941,
+      "step": 720
+    },
+    {
+      "epoch": 0.6339557099435519,
+      "grad_norm": 0.6526642441749573,
+      "learning_rate": 0.0002,
+      "loss": 0.4702,
+      "step": 730
+    },
+    {
+      "epoch": 0.6426400347372991,
+      "grad_norm": 0.7835870385169983,
+      "learning_rate": 0.0002,
+      "loss": 0.4646,
+      "step": 740
+    },
+    {
+      "epoch": 0.6513243595310465,
+      "grad_norm": 0.7449156641960144,
+      "learning_rate": 0.0002,
+      "loss": 0.456,
+      "step": 750
+    },
+    {
+      "epoch": 0.6600086843247938,
+      "grad_norm": 0.6138056516647339,
+      "learning_rate": 0.0002,
+      "loss": 0.4664,
+      "step": 760
+    },
+    {
+      "epoch": 0.668693009118541,
+      "grad_norm": 0.6755684614181519,
+      "learning_rate": 0.0002,
+      "loss": 0.4743,
+      "step": 770
+    },
+    {
+      "epoch": 0.6773773339122883,
+      "grad_norm": 0.7654828429222107,
+      "learning_rate": 0.0002,
+      "loss": 0.4624,
+      "step": 780
+    },
+    {
+      "epoch": 0.6860616587060356,
+      "grad_norm": 0.8211193084716797,
+      "learning_rate": 0.0002,
+      "loss": 0.482,
+      "step": 790
+    },
+    {
+      "epoch": 0.6947459834997829,
+      "grad_norm": 0.6197503805160522,
+      "learning_rate": 0.0002,
+      "loss": 0.4783,
+      "step": 800
+    },
+    {
+      "epoch": 0.7034303082935301,
+      "grad_norm": 0.8030904531478882,
+      "learning_rate": 0.0002,
+      "loss": 0.461,
+      "step": 810
+    },
+    {
+      "epoch": 0.7121146330872775,
+      "grad_norm": 0.5908632278442383,
+      "learning_rate": 0.0002,
+      "loss": 0.4726,
+      "step": 820
+    },
+    {
+      "epoch": 0.7207989578810248,
+      "grad_norm": 0.867847204208374,
+      "learning_rate": 0.0002,
+      "loss": 0.4683,
+      "step": 830
+    },
+    {
+      "epoch": 0.729483282674772,
+      "grad_norm": 0.8335438966751099,
+      "learning_rate": 0.0002,
+      "loss": 0.4494,
+      "step": 840
+    },
+    {
+      "epoch": 0.7381676074685193,
+      "grad_norm": 0.7276930212974548,
+      "learning_rate": 0.0002,
+      "loss": 0.4446,
+      "step": 850
+    },
+    {
+      "epoch": 0.7468519322622666,
+      "grad_norm": 0.6241617202758789,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 860
+    },
+    {
+      "epoch": 0.7555362570560139,
+      "grad_norm": 0.6197535395622253,
+      "learning_rate": 0.0002,
+      "loss": 0.4652,
+      "step": 870
+    },
+    {
+      "epoch": 0.7642205818497612,
+      "grad_norm": 0.7276974320411682,
+      "learning_rate": 0.0002,
+      "loss": 0.4518,
+      "step": 880
+    },
+    {
+      "epoch": 0.7729049066435084,
+      "grad_norm": 0.7494757771492004,
+      "learning_rate": 0.0002,
+      "loss": 0.4533,
+      "step": 890
+    },
+    {
+      "epoch": 0.7815892314372558,
+      "grad_norm": 0.6311206817626953,
+      "learning_rate": 0.0002,
+      "loss": 0.4534,
+      "step": 900
+    },
+    {
+      "epoch": 0.790273556231003,
+      "grad_norm": 0.6753235459327698,
+      "learning_rate": 0.0002,
+      "loss": 0.4435,
+      "step": 910
+    },
+    {
+      "epoch": 0.7989578810247503,
+      "grad_norm": 0.7800608277320862,
+      "learning_rate": 0.0002,
+      "loss": 0.4602,
+      "step": 920
+    },
+    {
+      "epoch": 0.8076422058184977,
+      "grad_norm": 0.6053752899169922,
+      "learning_rate": 0.0002,
+      "loss": 0.4502,
+      "step": 930
+    },
+    {
+      "epoch": 0.8163265306122449,
+      "grad_norm": 0.8059675097465515,
+      "learning_rate": 0.0002,
+      "loss": 0.4454,
+      "step": 940
+    },
+    {
+      "epoch": 0.8250108554059922,
+      "grad_norm": 0.7138864398002625,
+      "learning_rate": 0.0002,
+      "loss": 0.4626,
+      "step": 950
+    },
+    {
+      "epoch": 0.8336951801997394,
+      "grad_norm": 0.7769238948822021,
+      "learning_rate": 0.0002,
+      "loss": 0.4327,
+      "step": 960
+    },
+    {
+      "epoch": 0.8423795049934868,
+      "grad_norm": 0.7062289118766785,
+      "learning_rate": 0.0002,
+      "loss": 0.4478,
+      "step": 970
+    },
+    {
+      "epoch": 0.851063829787234,
+      "grad_norm": 0.744469940662384,
+      "learning_rate": 0.0002,
+      "loss": 0.4329,
+      "step": 980
+    },
+    {
+      "epoch": 0.8597481545809813,
+      "grad_norm": 0.6722288727760315,
+      "learning_rate": 0.0002,
+      "loss": 0.4303,
+      "step": 990
+    },
+    {
+      "epoch": 0.8684324793747286,
+      "grad_norm": 0.7886863946914673,
+      "learning_rate": 0.0002,
+      "loss": 0.4232,
+      "step": 1000
+    },
+    {
+      "epoch": 0.8771168041684759,
+      "grad_norm": 0.8501645922660828,
+      "learning_rate": 0.0002,
+      "loss": 0.4201,
+      "step": 1010
+    },
+    {
+      "epoch": 0.8858011289622232,
+      "grad_norm": 0.9484928250312805,
+      "learning_rate": 0.0002,
+      "loss": 0.4491,
+      "step": 1020
+    },
+    {
+      "epoch": 0.8944854537559704,
+      "grad_norm": 0.6376281976699829,
+      "learning_rate": 0.0002,
+      "loss": 0.4121,
+      "step": 1030
+    },
+    {
+      "epoch": 0.9031697785497178,
+      "grad_norm": 0.5941802263259888,
+      "learning_rate": 0.0002,
+      "loss": 0.4223,
+      "step": 1040
+    },
+    {
+      "epoch": 0.9118541033434651,
+      "grad_norm": 0.75499027967453,
+      "learning_rate": 0.0002,
+      "loss": 0.4319,
+      "step": 1050
+    },
+    {
+      "epoch": 0.9205384281372123,
+      "grad_norm": 0.8150404095649719,
+      "learning_rate": 0.0002,
+      "loss": 0.4286,
+      "step": 1060
+    },
+    {
+      "epoch": 0.9292227529309596,
+      "grad_norm": 0.8157607316970825,
+      "learning_rate": 0.0002,
+      "loss": 0.4379,
+      "step": 1070
+    },
+    {
+      "epoch": 0.9379070777247069,
+      "grad_norm": 0.6417438387870789,
+      "learning_rate": 0.0002,
+      "loss": 0.4459,
+      "step": 1080
+    },
+    {
+      "epoch": 0.9465914025184542,
+      "grad_norm": 0.8608890175819397,
+      "learning_rate": 0.0002,
+      "loss": 0.4161,
+      "step": 1090
+    },
+    {
+      "epoch": 0.9552757273122015,
+      "grad_norm": 0.5620037913322449,
+      "learning_rate": 0.0002,
+      "loss": 0.4155,
+      "step": 1100
+    },
+    {
+      "epoch": 0.9639600521059488,
+      "grad_norm": 1.3118562698364258,
+      "learning_rate": 0.0002,
+      "loss": 0.4215,
+      "step": 1110
+    },
+    {
+      "epoch": 0.9726443768996961,
+      "grad_norm": 0.8373771905899048,
+      "learning_rate": 0.0002,
+      "loss": 0.4017,
+      "step": 1120
+    },
+    {
+      "epoch": 0.9813287016934433,
+      "grad_norm": 0.7725268602371216,
+      "learning_rate": 0.0002,
+      "loss": 0.4014,
+      "step": 1130
+    },
+    {
+      "epoch": 0.9900130264871906,
+      "grad_norm": 0.7564753890037537,
+      "learning_rate": 0.0002,
+      "loss": 0.4143,
+      "step": 1140
+    },
+    {
+      "epoch": 0.998697351280938,
+      "grad_norm": 0.7307513952255249,
+      "learning_rate": 0.0002,
+      "loss": 0.4085,
+      "step": 1150
+    },
+    {
+      "epoch": 0.9995657837603127,
+      "eval_loss": 0.4137004315853119,
+      "eval_runtime": 51.9598,
+      "eval_samples_per_second": 7.025,
+      "eval_steps_per_second": 0.885,
+      "step": 1151
+    },
+    {
+      "epoch": 1.0073816760746852,
+      "grad_norm": 0.880082905292511,
+      "learning_rate": 0.0002,
+      "loss": 0.3848,
+      "step": 1160
+    },
+    {
+      "epoch": 1.0160660008684326,
+      "grad_norm": 0.7512372732162476,
+      "learning_rate": 0.0002,
+      "loss": 0.3722,
+      "step": 1170
+    },
+    {
+      "epoch": 1.0247503256621797,
+      "grad_norm": 0.6832528710365295,
+      "learning_rate": 0.0002,
+      "loss": 0.3756,
+      "step": 1180
+    },
+    {
+      "epoch": 1.033434650455927,
+      "grad_norm": 0.9188297986984253,
+      "learning_rate": 0.0002,
+      "loss": 0.3798,
+      "step": 1190
+    },
+    {
+      "epoch": 1.0421189752496742,
+      "grad_norm": 0.6144621968269348,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 1200
+    },
+    {
+      "epoch": 1.0508033000434216,
+      "grad_norm": 0.750042200088501,
+      "learning_rate": 0.0002,
+      "loss": 0.3893,
+      "step": 1210
+    },
+    {
+      "epoch": 1.059487624837169,
+      "grad_norm": 0.6084040999412537,
+      "learning_rate": 0.0002,
+      "loss": 0.3793,
+      "step": 1220
+    },
+    {
+      "epoch": 1.068171949630916,
+      "grad_norm": 0.8569493889808655,
+      "learning_rate": 0.0002,
+      "loss": 0.3857,
+      "step": 1230
+    },
+    {
+      "epoch": 1.0768562744246635,
+      "grad_norm": 0.8420213460922241,
+      "learning_rate": 0.0002,
+      "loss": 0.381,
+      "step": 1240
+    },
+    {
+      "epoch": 1.0855405992184108,
+      "grad_norm": 0.5716078281402588,
+      "learning_rate": 0.0002,
+      "loss": 0.3905,
+      "step": 1250
+    },
+    {
+      "epoch": 1.094224924012158,
+      "grad_norm": 0.6244675517082214,
+      "learning_rate": 0.0002,
+      "loss": 0.3578,
+      "step": 1260
+    },
+    {
+      "epoch": 1.1029092488059054,
+      "grad_norm": 0.7346081733703613,
+      "learning_rate": 0.0002,
+      "loss": 0.3817,
+      "step": 1270
+    },
+    {
+      "epoch": 1.1115935735996527,
+      "grad_norm": 1.0054655075073242,
+      "learning_rate": 0.0002,
+      "loss": 0.3738,
+      "step": 1280
+    },
+    {
+      "epoch": 1.1202778983933999,
+      "grad_norm": 0.7442952394485474,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 1290
+    },
+    {
+      "epoch": 1.1289622231871472,
+      "grad_norm": 0.7852597236633301,
+      "learning_rate": 0.0002,
+      "loss": 0.3996,
+      "step": 1300
+    },
+    {
+      "epoch": 1.1376465479808946,
+      "grad_norm": 0.9616802334785461,
+      "learning_rate": 0.0002,
+      "loss": 0.3878,
+      "step": 1310
+    },
+    {
+      "epoch": 1.1463308727746417,
+      "grad_norm": 0.7134007811546326,
+      "learning_rate": 0.0002,
+      "loss": 0.3747,
+      "step": 1320
+    },
+    {
+      "epoch": 1.155015197568389,
+      "grad_norm": 0.6189036965370178,
+      "learning_rate": 0.0002,
+      "loss": 0.3609,
+      "step": 1330
+    },
+    {
+      "epoch": 1.1636995223621363,
+      "grad_norm": 0.7725108861923218,
+      "learning_rate": 0.0002,
+      "loss": 0.3793,
+      "step": 1340
+    },
+    {
+      "epoch": 1.1723838471558836,
+      "grad_norm": 0.8503804802894592,
+      "learning_rate": 0.0002,
+      "loss": 0.3687,
+      "step": 1350
+    },
+    {
+      "epoch": 1.181068171949631,
+      "grad_norm": 0.7709351181983948,
+      "learning_rate": 0.0002,
+      "loss": 0.3767,
+      "step": 1360
+    },
+    {
+      "epoch": 1.1897524967433781,
+      "grad_norm": 0.9070065021514893,
+      "learning_rate": 0.0002,
+      "loss": 0.3914,
+      "step": 1370
+    },
+    {
+      "epoch": 1.1984368215371255,
+      "grad_norm": 0.7514732480049133,
+      "learning_rate": 0.0002,
+      "loss": 0.3962,
+      "step": 1380
+    },
+    {
+      "epoch": 1.2071211463308729,
+      "grad_norm": 0.9331297874450684,
+      "learning_rate": 0.0002,
+      "loss": 0.3597,
+      "step": 1390
+    },
+    {
+      "epoch": 1.21580547112462,
+      "grad_norm": 0.9618472456932068,
+      "learning_rate": 0.0002,
+      "loss": 0.3764,
+      "step": 1400
+    },
+    {
+      "epoch": 1.2244897959183674,
+      "grad_norm": 0.8956435918807983,
+      "learning_rate": 0.0002,
+      "loss": 0.3709,
+      "step": 1410
+    },
+    {
+      "epoch": 1.2331741207121145,
+      "grad_norm": 0.9150217175483704,
+      "learning_rate": 0.0002,
+      "loss": 0.3775,
+      "step": 1420
+    },
+    {
+      "epoch": 1.2418584455058619,
+      "grad_norm": 0.6232621073722839,
+      "learning_rate": 0.0002,
+      "loss": 0.3656,
+      "step": 1430
+    },
+    {
+      "epoch": 1.2505427702996093,
+      "grad_norm": 0.980364978313446,
+      "learning_rate": 0.0002,
+      "loss": 0.3831,
+      "step": 1440
+    },
+    {
+      "epoch": 1.2592270950933564,
+      "grad_norm": 0.8638973236083984,
+      "learning_rate": 0.0002,
+      "loss": 0.3514,
+      "step": 1450
+    },
+    {
+      "epoch": 1.2679114198871038,
+      "grad_norm": 0.6402583122253418,
+      "learning_rate": 0.0002,
+      "loss": 0.3659,
+      "step": 1460
+    },
+    {
+      "epoch": 1.2765957446808511,
+      "grad_norm": 0.9318592548370361,
+      "learning_rate": 0.0002,
+      "loss": 0.3772,
+      "step": 1470
+    },
+    {
+      "epoch": 1.2852800694745983,
+      "grad_norm": 0.7031506299972534,
+      "learning_rate": 0.0002,
+      "loss": 0.3727,
+      "step": 1480
+    },
+    {
+      "epoch": 1.2939643942683456,
+      "grad_norm": 0.9225306510925293,
+      "learning_rate": 0.0002,
+      "loss": 0.3615,
+      "step": 1490
+    },
+    {
+      "epoch": 1.302648719062093,
+      "grad_norm": 0.8340631127357483,
+      "learning_rate": 0.0002,
+      "loss": 0.3838,
+      "step": 1500
+    },
+    {
+      "epoch": 1.3113330438558402,
+      "grad_norm": 0.9431206583976746,
+      "learning_rate": 0.0002,
+      "loss": 0.3747,
+      "step": 1510
+    },
+    {
+      "epoch": 1.3200173686495875,
+      "grad_norm": 0.8804561495780945,
+      "learning_rate": 0.0002,
+      "loss": 0.3535,
+      "step": 1520
+    },
+    {
+      "epoch": 1.3287016934433349,
+      "grad_norm": 0.8626439571380615,
+      "learning_rate": 0.0002,
+      "loss": 0.3679,
+      "step": 1530
+    },
+    {
+      "epoch": 1.337386018237082,
+      "grad_norm": 0.8414568305015564,
+      "learning_rate": 0.0002,
+      "loss": 0.3644,
+      "step": 1540
+    },
+    {
+      "epoch": 1.3460703430308294,
+      "grad_norm": 1.127997636795044,
+      "learning_rate": 0.0002,
+      "loss": 0.3604,
+      "step": 1550
+    },
+    {
+      "epoch": 1.3547546678245768,
+      "grad_norm": 0.7714207172393799,
+      "learning_rate": 0.0002,
+      "loss": 0.3589,
+      "step": 1560
+    },
+    {
+      "epoch": 1.363438992618324,
+      "grad_norm": 0.9359990954399109,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1570
+    },
+    {
+      "epoch": 1.3721233174120713,
+      "grad_norm": 0.7754091620445251,
+      "learning_rate": 0.0002,
+      "loss": 0.3694,
+      "step": 1580
+    },
+    {
+      "epoch": 1.3808076422058184,
+      "grad_norm": 1.0162203311920166,
+      "learning_rate": 0.0002,
+      "loss": 0.36,
+      "step": 1590
+    },
+    {
+      "epoch": 1.3894919669995658,
+      "grad_norm": 0.9444917440414429,
+      "learning_rate": 0.0002,
+      "loss": 0.347,
+      "step": 1600
+    },
+    {
+      "epoch": 1.3981762917933132,
+      "grad_norm": 0.8122510313987732,
+      "learning_rate": 0.0002,
+      "loss": 0.3625,
+      "step": 1610
+    },
+    {
+      "epoch": 1.4068606165870603,
+      "grad_norm": 0.7946245670318604,
+      "learning_rate": 0.0002,
+      "loss": 0.3499,
+      "step": 1620
+    },
+    {
+      "epoch": 1.4155449413808077,
+      "grad_norm": 0.899692714214325,
+      "learning_rate": 0.0002,
+      "loss": 0.3606,
+      "step": 1630
+    },
+    {
+      "epoch": 1.4242292661745548,
+      "grad_norm": 0.652953565120697,
+      "learning_rate": 0.0002,
+      "loss": 0.3599,
+      "step": 1640
+    },
+    {
+      "epoch": 1.4329135909683022,
+      "grad_norm": 0.6942960023880005,
+      "learning_rate": 0.0002,
+      "loss": 0.3585,
+      "step": 1650
+    },
+    {
+      "epoch": 1.4415979157620495,
+      "grad_norm": 0.5737764239311218,
+      "learning_rate": 0.0002,
+      "loss": 0.3624,
+      "step": 1660
+    },
+    {
+      "epoch": 1.4502822405557967,
+      "grad_norm": 0.7966623306274414,
+      "learning_rate": 0.0002,
+      "loss": 0.3446,
+      "step": 1670
+    },
+    {
+      "epoch": 1.458966565349544,
+      "grad_norm": 0.7757281064987183,
+      "learning_rate": 0.0002,
+      "loss": 0.34,
+      "step": 1680
+    },
+    {
+      "epoch": 1.4676508901432914,
+      "grad_norm": 0.7565333843231201,
+      "learning_rate": 0.0002,
+      "loss": 0.377,
+      "step": 1690
+    },
+    {
+      "epoch": 1.4763352149370386,
+      "grad_norm": 1.2639003992080688,
+      "learning_rate": 0.0002,
+      "loss": 0.3536,
+      "step": 1700
+    },
+    {
+      "epoch": 1.485019539730786,
+      "grad_norm": 1.0366960763931274,
+      "learning_rate": 0.0002,
+      "loss": 0.3503,
+      "step": 1710
+    },
+    {
+      "epoch": 1.4937038645245333,
+      "grad_norm": 0.8083975911140442,
+      "learning_rate": 0.0002,
+      "loss": 0.3594,
+      "step": 1720
+    },
+    {
+      "epoch": 1.5023881893182804,
+      "grad_norm": 1.0527535676956177,
+      "learning_rate": 0.0002,
+      "loss": 0.369,
+      "step": 1730
+    },
+    {
+      "epoch": 1.5110725141120278,
+      "grad_norm": 0.7138310074806213,
+      "learning_rate": 0.0002,
+      "loss": 0.3424,
+      "step": 1740
+    },
+    {
+      "epoch": 1.5197568389057752,
+      "grad_norm": 0.818398118019104,
+      "learning_rate": 0.0002,
+      "loss": 0.3568,
+      "step": 1750
+    },
+    {
+      "epoch": 1.5284411636995223,
+      "grad_norm": 0.7518394589424133,
+      "learning_rate": 0.0002,
+      "loss": 0.3347,
+      "step": 1760
+    },
+    {
+      "epoch": 1.5371254884932697,
+      "grad_norm": 0.6814934611320496,
+      "learning_rate": 0.0002,
+      "loss": 0.3462,
+      "step": 1770
+    },
+    {
+      "epoch": 1.545809813287017,
+      "grad_norm": 0.49772709608078003,
+      "learning_rate": 0.0002,
+      "loss": 0.3432,
+      "step": 1780
+    },
+    {
+      "epoch": 1.5544941380807642,
+      "grad_norm": 0.7579216957092285,
+      "learning_rate": 0.0002,
+      "loss": 0.3457,
+      "step": 1790
+    },
+    {
+      "epoch": 1.5631784628745113,
+      "grad_norm": 0.7244242429733276,
+      "learning_rate": 0.0002,
+      "loss": 0.3517,
+      "step": 1800
+    },
+    {
+      "epoch": 1.571862787668259,
+      "grad_norm": 0.6358800530433655,
+      "learning_rate": 0.0002,
+      "loss": 0.3427,
+      "step": 1810
+    },
+    {
+      "epoch": 1.580547112462006,
+      "grad_norm": 0.6636850833892822,
+      "learning_rate": 0.0002,
+      "loss": 0.3482,
+      "step": 1820
+    },
+    {
+      "epoch": 1.5892314372557532,
+      "grad_norm": 0.691331148147583,
+      "learning_rate": 0.0002,
+      "loss": 0.3397,
+      "step": 1830
+    },
+    {
+      "epoch": 1.5979157620495008,
+      "grad_norm": 0.8501580357551575,
+      "learning_rate": 0.0002,
+      "loss": 0.3525,
+      "step": 1840
+    },
+    {
+      "epoch": 1.606600086843248,
+      "grad_norm": 0.8448147177696228,
+      "learning_rate": 0.0002,
+      "loss": 0.3319,
+      "step": 1850
+    },
+    {
+      "epoch": 1.615284411636995,
+      "grad_norm": 0.6357255578041077,
+      "learning_rate": 0.0002,
+      "loss": 0.3355,
+      "step": 1860
+    },
+    {
+      "epoch": 1.6239687364307425,
+      "grad_norm": 1.008238673210144,
+      "learning_rate": 0.0002,
+      "loss": 0.3592,
+      "step": 1870
+    },
+    {
+      "epoch": 1.6326530612244898,
+      "grad_norm": 0.7839747667312622,
+      "learning_rate": 0.0002,
+      "loss": 0.3342,
+      "step": 1880
+    },
+    {
+      "epoch": 1.641337386018237,
+      "grad_norm": 1.1214498281478882,
+      "learning_rate": 0.0002,
+      "loss": 0.3348,
+      "step": 1890
+    },
+    {
+      "epoch": 1.6500217108119843,
+      "grad_norm": 0.9457558989524841,
+      "learning_rate": 0.0002,
+      "loss": 0.335,
+      "step": 1900
+    },
+    {
+      "epoch": 1.6587060356057317,
+      "grad_norm": 0.8412042260169983,
+      "learning_rate": 0.0002,
+      "loss": 0.3425,
+      "step": 1910
+    },
+    {
+      "epoch": 1.6673903603994789,
+      "grad_norm": 1.0219208002090454,
+      "learning_rate": 0.0002,
+      "loss": 0.3427,
+      "step": 1920
+    },
+    {
+      "epoch": 1.6760746851932262,
+      "grad_norm": 1.079763412475586,
+      "learning_rate": 0.0002,
+      "loss": 0.3429,
+      "step": 1930
+    },
+    {
+      "epoch": 1.6847590099869736,
+      "grad_norm": 0.6026722192764282,
+      "learning_rate": 0.0002,
+      "loss": 0.3433,
+      "step": 1940
+    },
+    {
+      "epoch": 1.6934433347807207,
+      "grad_norm": 0.840270459651947,
+      "learning_rate": 0.0002,
+      "loss": 0.3438,
+      "step": 1950
+    },
+    {
+      "epoch": 1.702127659574468,
+      "grad_norm": 0.8331791758537292,
+      "learning_rate": 0.0002,
+      "loss": 0.3544,
+      "step": 1960
+    },
+    {
+      "epoch": 1.7108119843682155,
+      "grad_norm": 0.6232297420501709,
+      "learning_rate": 0.0002,
+      "loss": 0.3396,
+      "step": 1970
+    },
+    {
+      "epoch": 1.7194963091619626,
+      "grad_norm": 0.8575964570045471,
+      "learning_rate": 0.0002,
+      "loss": 0.3413,
+      "step": 1980
+    },
+    {
+      "epoch": 1.72818063395571,
+      "grad_norm": 1.4170857667922974,
+      "learning_rate": 0.0002,
+      "loss": 0.3375,
+      "step": 1990
+    },
+    {
+      "epoch": 1.7368649587494573,
+      "grad_norm": 0.7481379508972168,
+      "learning_rate": 0.0002,
+      "loss": 0.3328,
+      "step": 2000
+    },
+    {
+      "epoch": 1.7455492835432045,
+      "grad_norm": 0.6883425712585449,
+      "learning_rate": 0.0002,
+      "loss": 0.3413,
+      "step": 2010
+    },
+    {
+      "epoch": 1.7542336083369519,
+      "grad_norm": 1.1543880701065063,
+      "learning_rate": 0.0002,
+      "loss": 0.3381,
+      "step": 2020
+    },
+    {
+      "epoch": 1.7629179331306992,
+      "grad_norm": 0.8788072466850281,
+      "learning_rate": 0.0002,
+      "loss": 0.3369,
+      "step": 2030
+    },
+    {
+      "epoch": 1.7716022579244464,
+      "grad_norm": 0.7034786343574524,
+      "learning_rate": 0.0002,
+      "loss": 0.3266,
+      "step": 2040
+    },
+    {
+      "epoch": 1.7802865827181935,
+      "grad_norm": 1.041170358657837,
+      "learning_rate": 0.0002,
+      "loss": 0.3263,
+      "step": 2050
+    },
+    {
+      "epoch": 1.788970907511941,
+      "grad_norm": 0.7966195344924927,
+      "learning_rate": 0.0002,
+      "loss": 0.3309,
+      "step": 2060
+    },
+    {
+      "epoch": 1.7976552323056882,
+      "grad_norm": 0.8562559485435486,
+      "learning_rate": 0.0002,
+      "loss": 0.3174,
+      "step": 2070
+    },
+    {
+      "epoch": 1.8063395570994354,
+      "grad_norm": 0.9997917413711548,
+      "learning_rate": 0.0002,
+      "loss": 0.3384,
+      "step": 2080
+    },
+    {
+      "epoch": 1.8150238818931828,
+      "grad_norm": 0.9915640950202942,
+      "learning_rate": 0.0002,
+      "loss": 0.3291,
+      "step": 2090
+    },
+    {
+      "epoch": 1.8237082066869301,
+      "grad_norm": 0.8850535154342651,
+      "learning_rate": 0.0002,
+      "loss": 0.3266,
+      "step": 2100
+    },
+    {
+      "epoch": 1.8323925314806773,
+      "grad_norm": 0.7072052359580994,
+      "learning_rate": 0.0002,
+      "loss": 0.3269,
+      "step": 2110
+    },
+    {
+      "epoch": 1.8410768562744246,
+      "grad_norm": 0.8423166871070862,
+      "learning_rate": 0.0002,
+      "loss": 0.3302,
+      "step": 2120
+    },
+    {
+      "epoch": 1.849761181068172,
+      "grad_norm": 0.7291885614395142,
+      "learning_rate": 0.0002,
+      "loss": 0.3382,
+      "step": 2130
+    },
+    {
+      "epoch": 1.8584455058619191,
+      "grad_norm": 1.0388280153274536,
+      "learning_rate": 0.0002,
+      "loss": 0.3247,
+      "step": 2140
+    },
+    {
+      "epoch": 1.8671298306556665,
+      "grad_norm": 0.8792893886566162,
+      "learning_rate": 0.0002,
+      "loss": 0.3223,
+      "step": 2150
+    },
+    {
+      "epoch": 1.8758141554494139,
+      "grad_norm": 0.852391242980957,
+      "learning_rate": 0.0002,
+      "loss": 0.3385,
+      "step": 2160
+    },
+    {
+      "epoch": 1.884498480243161,
+      "grad_norm": 0.7536265850067139,
+      "learning_rate": 0.0002,
+      "loss": 0.3253,
+      "step": 2170
+    },
+    {
+      "epoch": 1.8931828050369084,
+      "grad_norm": 0.8531852960586548,
+      "learning_rate": 0.0002,
+      "loss": 0.3234,
+      "step": 2180
+    },
+    {
+      "epoch": 1.9018671298306558,
+      "grad_norm": 0.6752887964248657,
+      "learning_rate": 0.0002,
+      "loss": 0.3352,
+      "step": 2190
+    },
+    {
+      "epoch": 1.910551454624403,
+      "grad_norm": 0.9746645092964172,
+      "learning_rate": 0.0002,
+      "loss": 0.3344,
+      "step": 2200
+    },
+    {
+      "epoch": 1.9192357794181503,
+      "grad_norm": 0.8092345595359802,
+      "learning_rate": 0.0002,
+      "loss": 0.331,
+      "step": 2210
+    },
+    {
+      "epoch": 1.9279201042118976,
+      "grad_norm": 0.6521363854408264,
+      "learning_rate": 0.0002,
+      "loss": 0.3245,
+      "step": 2220
+    },
+    {
+      "epoch": 1.9366044290056448,
+      "grad_norm": 1.0131723880767822,
+      "learning_rate": 0.0002,
+      "loss": 0.3272,
+      "step": 2230
+    },
+    {
+      "epoch": 1.9452887537993921,
+      "grad_norm": 0.6056963205337524,
+      "learning_rate": 0.0002,
+      "loss": 0.3166,
+      "step": 2240
+    },
+    {
+      "epoch": 1.9539730785931395,
+      "grad_norm": 0.9994075894355774,
+      "learning_rate": 0.0002,
+      "loss": 0.3346,
+      "step": 2250
+    },
+    {
+      "epoch": 1.9626574033868867,
+      "grad_norm": 0.8796638250350952,
+      "learning_rate": 0.0002,
+      "loss": 0.3214,
+      "step": 2260
+    },
+    {
+      "epoch": 1.9713417281806338,
+      "grad_norm": 0.6505881547927856,
+      "learning_rate": 0.0002,
+      "loss": 0.3225,
+      "step": 2270
+    },
+    {
+      "epoch": 1.9800260529743814,
+      "grad_norm": 1.1708468198776245,
+      "learning_rate": 0.0002,
+      "loss": 0.3346,
+      "step": 2280
+    },
+    {
+      "epoch": 1.9887103777681285,
+      "grad_norm": 0.898364245891571,
+      "learning_rate": 0.0002,
+      "loss": 0.3234,
+      "step": 2290
+    },
+    {
+      "epoch": 1.9973947025618757,
+      "grad_norm": 0.7017287611961365,
+      "learning_rate": 0.0002,
+      "loss": 0.3263,
+      "step": 2300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.3223828971385956,
+      "eval_runtime": 51.2747,
+      "eval_samples_per_second": 7.119,
+      "eval_steps_per_second": 0.897,
+      "step": 2303
+    },
+    {
+      "epoch": 2.0060790273556233,
+      "grad_norm": 0.807558000087738,
+      "learning_rate": 0.0002,
+      "loss": 0.3249,
+      "step": 2310
+    },
+    {
+      "epoch": 2.0147633521493704,
+      "grad_norm": 1.107229471206665,
+      "learning_rate": 0.0002,
+      "loss": 0.3036,
+      "step": 2320
+    },
+    {
+      "epoch": 2.0234476769431176,
+      "grad_norm": 0.8265573978424072,
+      "learning_rate": 0.0002,
+      "loss": 0.3057,
+      "step": 2330
+    },
+    {
+      "epoch": 2.032132001736865,
+      "grad_norm": 0.6224291324615479,
+      "learning_rate": 0.0002,
+      "loss": 0.3073,
+      "step": 2340
+    },
+    {
+      "epoch": 2.0408163265306123,
+      "grad_norm": 0.8967212438583374,
+      "learning_rate": 0.0002,
+      "loss": 0.3086,
+      "step": 2350
+    },
+    {
+      "epoch": 2.0495006513243594,
+      "grad_norm": 0.6476389765739441,
+      "learning_rate": 0.0002,
+      "loss": 0.3059,
+      "step": 2360
+    },
+    {
+      "epoch": 2.058184976118107,
+      "grad_norm": 0.7838577628135681,
+      "learning_rate": 0.0002,
+      "loss": 0.3013,
+      "step": 2370
+    },
+    {
+      "epoch": 2.066869300911854,
+      "grad_norm": 0.9126287698745728,
+      "learning_rate": 0.0002,
+      "loss": 0.3136,
+      "step": 2380
+    },
+    {
+      "epoch": 2.0755536257056013,
+      "grad_norm": 0.946692705154419,
+      "learning_rate": 0.0002,
+      "loss": 0.3007,
+      "step": 2390
+    },
+    {
+      "epoch": 2.0842379504993485,
+      "grad_norm": 0.6643753051757812,
+      "learning_rate": 0.0002,
+      "loss": 0.2997,
+      "step": 2400
+    },
+    {
+      "epoch": 2.092922275293096,
+      "grad_norm": 0.7992640733718872,
+      "learning_rate": 0.0002,
+      "loss": 0.3077,
+      "step": 2410
+    },
+    {
+      "epoch": 2.101606600086843,
+      "grad_norm": 1.0821648836135864,
+      "learning_rate": 0.0002,
+      "loss": 0.3142,
+      "step": 2420
+    },
+    {
+      "epoch": 2.1102909248805903,
+      "grad_norm": 0.7750299572944641,
+      "learning_rate": 0.0002,
+      "loss": 0.3019,
+      "step": 2430
+    },
+    {
+      "epoch": 2.118975249674338,
+      "grad_norm": 0.8424153327941895,
+      "learning_rate": 0.0002,
+      "loss": 0.3059,
+      "step": 2440
+    },
+    {
+      "epoch": 2.127659574468085,
+      "grad_norm": 1.0579770803451538,
+      "learning_rate": 0.0002,
+      "loss": 0.301,
+      "step": 2450
+    },
+    {
+      "epoch": 2.136343899261832,
+      "grad_norm": 0.7864382863044739,
+      "learning_rate": 0.0002,
+      "loss": 0.3061,
+      "step": 2460
+    },
+    {
+      "epoch": 2.14502822405558,
+      "grad_norm": 0.8209099173545837,
+      "learning_rate": 0.0002,
+      "loss": 0.3055,
+      "step": 2470
+    },
+    {
+      "epoch": 2.153712548849327,
+      "grad_norm": 0.7030050754547119,
+      "learning_rate": 0.0002,
+      "loss": 0.2991,
+      "step": 2480
+    },
+    {
+      "epoch": 2.162396873643074,
+      "grad_norm": 0.8806471824645996,
+      "learning_rate": 0.0002,
+      "loss": 0.3212,
+      "step": 2490
+    },
+    {
+      "epoch": 2.1710811984368217,
+      "grad_norm": 0.7312551736831665,
+      "learning_rate": 0.0002,
+      "loss": 0.2991,
+      "step": 2500
+    },
+    {
+      "epoch": 2.179765523230569,
+      "grad_norm": 0.9340773224830627,
+      "learning_rate": 0.0002,
+      "loss": 0.3078,
+      "step": 2510
+    },
+    {
+      "epoch": 2.188449848024316,
+      "grad_norm": 0.633319079875946,
+      "learning_rate": 0.0002,
+      "loss": 0.3021,
+      "step": 2520
+    },
+    {
+      "epoch": 2.1971341728180636,
+      "grad_norm": 0.8805091381072998,
+      "learning_rate": 0.0002,
+      "loss": 0.3071,
+      "step": 2530
+    },
+    {
+      "epoch": 2.2058184976118107,
+      "grad_norm": 0.8670169115066528,
+      "learning_rate": 0.0002,
+      "loss": 0.2959,
+      "step": 2540
+    },
+    {
+      "epoch": 2.214502822405558,
+      "grad_norm": 0.8338320851325989,
+      "learning_rate": 0.0002,
+      "loss": 0.3025,
+      "step": 2550
+    },
+    {
+      "epoch": 2.2231871471993054,
+      "grad_norm": 0.9541687369346619,
+      "learning_rate": 0.0002,
+      "loss": 0.3024,
+      "step": 2560
+    },
+    {
+      "epoch": 2.2318714719930526,
+      "grad_norm": 0.976690948009491,
+      "learning_rate": 0.0002,
+      "loss": 0.3174,
+      "step": 2570
+    },
+    {
+      "epoch": 2.2405557967867997,
+      "grad_norm": 0.5785702466964722,
+      "learning_rate": 0.0002,
+      "loss": 0.3036,
+      "step": 2580
+    },
+    {
+      "epoch": 2.2492401215805473,
+      "grad_norm": 0.7772279381752014,
+      "learning_rate": 0.0002,
+      "loss": 0.3054,
+      "step": 2590
+    },
+    {
+      "epoch": 2.2579244463742945,
+      "grad_norm": 0.7839940786361694,
+      "learning_rate": 0.0002,
+      "loss": 0.3062,
+      "step": 2600
+    },
+    {
+      "epoch": 2.2666087711680416,
+      "grad_norm": 0.8164472579956055,
+      "learning_rate": 0.0002,
+      "loss": 0.2952,
+      "step": 2610
+    },
+    {
+      "epoch": 2.275293095961789,
+      "grad_norm": 0.9164624214172363,
+      "learning_rate": 0.0002,
+      "loss": 0.3078,
+      "step": 2620
+    },
+    {
+      "epoch": 2.2839774207555363,
+      "grad_norm": 1.0616720914840698,
+      "learning_rate": 0.0002,
+      "loss": 0.309,
+      "step": 2630
+    },
+    {
+      "epoch": 2.2926617455492835,
+      "grad_norm": 0.6295320391654968,
+      "learning_rate": 0.0002,
+      "loss": 0.307,
+      "step": 2640
+    },
+    {
+      "epoch": 2.3013460703430306,
+      "grad_norm": 0.6153804063796997,
+      "learning_rate": 0.0002,
+      "loss": 0.3003,
+      "step": 2650
+    },
+    {
+      "epoch": 2.310030395136778,
+      "grad_norm": 0.6307553648948669,
+      "learning_rate": 0.0002,
+      "loss": 0.3054,
+      "step": 2660
+    },
+    {
+      "epoch": 2.3187147199305254,
+      "grad_norm": 0.7876275777816772,
+      "learning_rate": 0.0002,
+      "loss": 0.2996,
+      "step": 2670
+    },
+    {
+      "epoch": 2.3273990447242725,
+      "grad_norm": 1.0939222574234009,
+      "learning_rate": 0.0002,
+      "loss": 0.3039,
+      "step": 2680
+    },
+    {
+      "epoch": 2.33608336951802,
+      "grad_norm": 0.755632758140564,
+      "learning_rate": 0.0002,
+      "loss": 0.3038,
+      "step": 2690
+    },
+    {
+      "epoch": 2.3447676943117672,
+      "grad_norm": 1.750001311302185,
+      "learning_rate": 0.0002,
+      "loss": 0.3009,
+      "step": 2700
+    },
+    {
+      "epoch": 2.3534520191055144,
+      "grad_norm": 0.6993144750595093,
+      "learning_rate": 0.0002,
+      "loss": 0.3077,
+      "step": 2710
+    },
+    {
+      "epoch": 2.362136343899262,
+      "grad_norm": 0.8878545165061951,
+      "learning_rate": 0.0002,
+      "loss": 0.2952,
+      "step": 2720
+    },
+    {
+      "epoch": 2.370820668693009,
+      "grad_norm": 0.737571656703949,
+      "learning_rate": 0.0002,
+      "loss": 0.3074,
+      "step": 2730
+    },
+    {
+      "epoch": 2.3795049934867563,
+      "grad_norm": 0.5662615299224854,
+      "learning_rate": 0.0002,
+      "loss": 0.2937,
+      "step": 2740
+    },
+    {
+      "epoch": 2.388189318280504,
+      "grad_norm": 1.082109808921814,
+      "learning_rate": 0.0002,
+      "loss": 0.2977,
+      "step": 2750
+    },
+    {
+      "epoch": 2.396873643074251,
+      "grad_norm": 0.761906623840332,
+      "learning_rate": 0.0002,
+      "loss": 0.2955,
+      "step": 2760
+    },
+    {
+      "epoch": 2.405557967867998,
+      "grad_norm": 1.020167350769043,
+      "learning_rate": 0.0002,
+      "loss": 0.3001,
+      "step": 2770
+    },
+    {
+      "epoch": 2.4142422926617457,
+      "grad_norm": 1.0099780559539795,
+      "learning_rate": 0.0002,
+      "loss": 0.3017,
+      "step": 2780
+    },
+    {
+      "epoch": 2.422926617455493,
+      "grad_norm": 1.0733739137649536,
+      "learning_rate": 0.0002,
+      "loss": 0.2991,
+      "step": 2790
+    },
+    {
+      "epoch": 2.43161094224924,
+      "grad_norm": 0.7780752778053284,
+      "learning_rate": 0.0002,
+      "loss": 0.309,
+      "step": 2800
+    },
+    {
+      "epoch": 2.4402952670429876,
+      "grad_norm": 0.8345295786857605,
+      "learning_rate": 0.0002,
+      "loss": 0.2924,
+      "step": 2810
+    },
+    {
+      "epoch": 2.4489795918367347,
+      "grad_norm": 0.7011855840682983,
+      "learning_rate": 0.0002,
+      "loss": 0.3052,
+      "step": 2820
+    },
+    {
+      "epoch": 2.457663916630482,
+      "grad_norm": 0.7993234992027283,
+      "learning_rate": 0.0002,
+      "loss": 0.3101,
+      "step": 2830
+    },
+    {
+      "epoch": 2.466348241424229,
+      "grad_norm": 0.7405799627304077,
+      "learning_rate": 0.0002,
+      "loss": 0.3067,
+      "step": 2840
+    },
+    {
+      "epoch": 2.4750325662179766,
+      "grad_norm": 0.6736032962799072,
+      "learning_rate": 0.0002,
+      "loss": 0.3017,
+      "step": 2850
+    },
+    {
+      "epoch": 2.4837168910117238,
+      "grad_norm": 0.6662697196006775,
+      "learning_rate": 0.0002,
+      "loss": 0.2991,
+      "step": 2860
+    },
+    {
+      "epoch": 2.4924012158054714,
+      "grad_norm": 0.7334403991699219,
+      "learning_rate": 0.0002,
+      "loss": 0.3047,
+      "step": 2870
+    },
+    {
+      "epoch": 2.5010855405992185,
+      "grad_norm": 1.1669126749038696,
+      "learning_rate": 0.0002,
+      "loss": 0.2906,
+      "step": 2880
+    },
+    {
+      "epoch": 2.5097698653929656,
+      "grad_norm": 0.8848074674606323,
+      "learning_rate": 0.0002,
+      "loss": 0.2967,
+      "step": 2890
+    },
+    {
+      "epoch": 2.518454190186713,
+      "grad_norm": 0.8354884386062622,
+      "learning_rate": 0.0002,
+      "loss": 0.3009,
+      "step": 2900
+    },
+    {
+      "epoch": 2.5271385149804604,
+      "grad_norm": 0.734761655330658,
+      "learning_rate": 0.0002,
+      "loss": 0.2907,
+      "step": 2910
+    },
+    {
+      "epoch": 2.5358228397742075,
+      "grad_norm": 0.73737633228302,
+      "learning_rate": 0.0002,
+      "loss": 0.2902,
+      "step": 2920
+    },
+    {
+      "epoch": 2.544507164567955,
+      "grad_norm": 1.2551928758621216,
+      "learning_rate": 0.0002,
+      "loss": 0.3023,
+      "step": 2930
+    },
+    {
+      "epoch": 2.5531914893617023,
+      "grad_norm": 0.8526908755302429,
+      "learning_rate": 0.0002,
+      "loss": 0.2978,
+      "step": 2940
+    },
+    {
+      "epoch": 2.5618758141554494,
+      "grad_norm": 0.7598440051078796,
+      "learning_rate": 0.0002,
+      "loss": 0.2998,
+      "step": 2950
+    },
+    {
+      "epoch": 2.5705601389491965,
+      "grad_norm": 0.4928401708602905,
+      "learning_rate": 0.0002,
+      "loss": 0.2927,
+      "step": 2960
+    },
+    {
+      "epoch": 2.579244463742944,
+      "grad_norm": 0.8329664468765259,
+      "learning_rate": 0.0002,
+      "loss": 0.2922,
+      "step": 2970
+    },
+    {
+      "epoch": 2.5879287885366913,
+      "grad_norm": 0.7364818453788757,
+      "learning_rate": 0.0002,
+      "loss": 0.2982,
+      "step": 2980
+    },
+    {
+      "epoch": 2.5966131133304384,
+      "grad_norm": 0.742920994758606,
+      "learning_rate": 0.0002,
+      "loss": 0.299,
+      "step": 2990
+    },
+    {
+      "epoch": 2.605297438124186,
+      "grad_norm": 0.6937646865844727,
+      "learning_rate": 0.0002,
+      "loss": 0.2963,
+      "step": 3000
+    },
+    {
+      "epoch": 2.613981762917933,
+      "grad_norm": 0.7259171009063721,
+      "learning_rate": 0.0002,
+      "loss": 0.2918,
+      "step": 3010
+    },
+    {
+      "epoch": 2.6226660877116803,
+      "grad_norm": 0.6129646301269531,
+      "learning_rate": 0.0002,
+      "loss": 0.2866,
+      "step": 3020
+    },
+    {
+      "epoch": 2.6313504125054274,
+      "grad_norm": 0.6982302069664001,
+      "learning_rate": 0.0002,
+      "loss": 0.29,
+      "step": 3030
+    },
+    {
+      "epoch": 2.640034737299175,
+      "grad_norm": 1.272072434425354,
+      "learning_rate": 0.0002,
+      "loss": 0.2968,
+      "step": 3040
+    },
+    {
+      "epoch": 2.648719062092922,
+      "grad_norm": 1.1105331182479858,
+      "learning_rate": 0.0002,
+      "loss": 0.3071,
+      "step": 3050
+    },
+    {
+      "epoch": 2.6574033868866698,
+      "grad_norm": 0.7434864044189453,
+      "learning_rate": 0.0002,
+      "loss": 0.2865,
+      "step": 3060
+    },
+    {
+      "epoch": 2.666087711680417,
+      "grad_norm": 0.8283593654632568,
+      "learning_rate": 0.0002,
+      "loss": 0.2973,
+      "step": 3070
+    },
+    {
+      "epoch": 2.674772036474164,
+      "grad_norm": 0.6259397268295288,
+      "learning_rate": 0.0002,
+      "loss": 0.292,
+      "step": 3080
+    },
+    {
+      "epoch": 2.683456361267911,
+      "grad_norm": 0.8509077429771423,
+      "learning_rate": 0.0002,
+      "loss": 0.29,
+      "step": 3090
+    },
+    {
+      "epoch": 2.692140686061659,
+      "grad_norm": 0.8883143067359924,
+      "learning_rate": 0.0002,
+      "loss": 0.2926,
+      "step": 3100
+    },
+    {
+      "epoch": 2.700825010855406,
+      "grad_norm": 0.7960503697395325,
+      "learning_rate": 0.0002,
+      "loss": 0.2928,
+      "step": 3110
+    },
+    {
+      "epoch": 2.7095093356491535,
+      "grad_norm": 0.7752923965454102,
+      "learning_rate": 0.0002,
+      "loss": 0.2935,
+      "step": 3120
+    },
+    {
+      "epoch": 2.7181936604429007,
+      "grad_norm": 0.6954398155212402,
+      "learning_rate": 0.0002,
+      "loss": 0.2819,
+      "step": 3130
+    },
+    {
+      "epoch": 2.726877985236648,
+      "grad_norm": 0.9302791953086853,
+      "learning_rate": 0.0002,
+      "loss": 0.2901,
+      "step": 3140
+    },
+    {
+      "epoch": 2.735562310030395,
+      "grad_norm": 0.7394571304321289,
+      "learning_rate": 0.0002,
+      "loss": 0.2941,
+      "step": 3150
+    },
+    {
+      "epoch": 2.7442466348241426,
+      "grad_norm": 0.7749691009521484,
+      "learning_rate": 0.0002,
+      "loss": 0.2865,
+      "step": 3160
+    },
+    {
+      "epoch": 2.7529309596178897,
+      "grad_norm": 0.8289293050765991,
+      "learning_rate": 0.0002,
+      "loss": 0.2906,
+      "step": 3170
+    },
+    {
+      "epoch": 2.761615284411637,
+      "grad_norm": 0.7008669376373291,
+      "learning_rate": 0.0002,
+      "loss": 0.2923,
+      "step": 3180
+    },
+    {
+      "epoch": 2.7702996092053844,
+      "grad_norm": 1.2574214935302734,
+      "learning_rate": 0.0002,
+      "loss": 0.2919,
+      "step": 3190
+    },
+    {
+      "epoch": 2.7789839339991316,
+      "grad_norm": 0.6361944675445557,
+      "learning_rate": 0.0002,
+      "loss": 0.29,
+      "step": 3200
+    },
+    {
+      "epoch": 2.7876682587928787,
+      "grad_norm": 0.6270219087600708,
+      "learning_rate": 0.0002,
+      "loss": 0.2848,
+      "step": 3210
+    },
+    {
+      "epoch": 2.7963525835866263,
+      "grad_norm": 0.5672947764396667,
+      "learning_rate": 0.0002,
+      "loss": 0.2895,
+      "step": 3220
+    },
+    {
+      "epoch": 2.8050369083803735,
+      "grad_norm": 0.8151955008506775,
+      "learning_rate": 0.0002,
+      "loss": 0.287,
+      "step": 3230
+    },
+    {
+      "epoch": 2.8137212331741206,
+      "grad_norm": 0.7757666110992432,
+      "learning_rate": 0.0002,
+      "loss": 0.295,
+      "step": 3240
+    },
+    {
+      "epoch": 2.822405557967868,
+      "grad_norm": 0.9094030857086182,
+      "learning_rate": 0.0002,
+      "loss": 0.2971,
+      "step": 3250
+    },
+    {
+      "epoch": 2.8310898827616153,
+      "grad_norm": 0.6486825346946716,
+      "learning_rate": 0.0002,
+      "loss": 0.2935,
+      "step": 3260
+    },
+    {
+      "epoch": 2.8397742075553625,
+      "grad_norm": 0.8660843968391418,
+      "learning_rate": 0.0002,
+      "loss": 0.288,
+      "step": 3270
+    },
+    {
+      "epoch": 2.8484585323491096,
+      "grad_norm": 0.6846482753753662,
+      "learning_rate": 0.0002,
+      "loss": 0.2785,
+      "step": 3280
+    },
+    {
+      "epoch": 2.857142857142857,
+      "grad_norm": 0.8217440247535706,
+      "learning_rate": 0.0002,
+      "loss": 0.2912,
+      "step": 3290
+    },
+    {
+      "epoch": 2.8658271819366044,
+      "grad_norm": 0.9265017509460449,
+      "learning_rate": 0.0002,
+      "loss": 0.2945,
+      "step": 3300
+    },
+    {
+      "epoch": 2.874511506730352,
+      "grad_norm": 0.6155156493186951,
+      "learning_rate": 0.0002,
+      "loss": 0.2967,
+      "step": 3310
+    },
+    {
+      "epoch": 2.883195831524099,
+      "grad_norm": 0.711634635925293,
+      "learning_rate": 0.0002,
+      "loss": 0.2922,
+      "step": 3320
+    },
+    {
+      "epoch": 2.8918801563178462,
+      "grad_norm": 0.7138526439666748,
+      "learning_rate": 0.0002,
+      "loss": 0.2829,
+      "step": 3330
+    },
+    {
+      "epoch": 2.9005644811115934,
+      "grad_norm": 0.8107818365097046,
+      "learning_rate": 0.0002,
+      "loss": 0.29,
+      "step": 3340
+    },
+    {
+      "epoch": 2.909248805905341,
+      "grad_norm": 1.169446587562561,
+      "learning_rate": 0.0002,
+      "loss": 0.2968,
+      "step": 3350
+    },
+    {
+      "epoch": 2.917933130699088,
+      "grad_norm": 0.8311055898666382,
+      "learning_rate": 0.0002,
+      "loss": 0.2844,
+      "step": 3360
+    },
+    {
+      "epoch": 2.9266174554928357,
+      "grad_norm": 0.741226077079773,
+      "learning_rate": 0.0002,
+      "loss": 0.2861,
+      "step": 3370
+    },
+    {
+      "epoch": 2.935301780286583,
+      "grad_norm": 0.6655572652816772,
+      "learning_rate": 0.0002,
+      "loss": 0.2861,
+      "step": 3380
+    },
+    {
+      "epoch": 2.94398610508033,
+      "grad_norm": 0.4534182846546173,
+      "learning_rate": 0.0002,
+      "loss": 0.2813,
+      "step": 3390
+    },
+    {
+      "epoch": 2.952670429874077,
+      "grad_norm": 0.8443124294281006,
+      "learning_rate": 0.0002,
+      "loss": 0.2899,
+      "step": 3400
+    },
+    {
+      "epoch": 2.9613547546678247,
+      "grad_norm": 0.748831570148468,
+      "learning_rate": 0.0002,
+      "loss": 0.2811,
+      "step": 3410
+    },
+    {
+      "epoch": 2.970039079461572,
+      "grad_norm": 0.6330915689468384,
+      "learning_rate": 0.0002,
+      "loss": 0.2842,
+      "step": 3420
+    },
+    {
+      "epoch": 2.978723404255319,
+      "grad_norm": 0.6212947964668274,
+      "learning_rate": 0.0002,
+      "loss": 0.2962,
+      "step": 3430
+    },
+    {
+      "epoch": 2.9874077290490666,
+      "grad_norm": 0.9319456815719604,
+      "learning_rate": 0.0002,
+      "loss": 0.2733,
+      "step": 3440
+    },
+    {
+      "epoch": 2.9960920538428137,
+      "grad_norm": 0.8269906044006348,
+      "learning_rate": 0.0002,
+      "loss": 0.2846,
+      "step": 3450
+    },
+    {
+      "epoch": 2.9995657837603127,
+      "eval_loss": 0.2924337387084961,
+      "eval_runtime": 51.4614,
+      "eval_samples_per_second": 7.093,
+      "eval_steps_per_second": 0.894,
+      "step": 3454
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 9208,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.515652434100224e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-3454/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e5ef1c767c8b99de039fadb161f732b5195c895194f4e546914308c1708a156
+size 5560

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b53a441325e38ec0a19a0e65b4785061ab65bc645f91329ce628bf25c9014e90
+size 109069176

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:526c929494f01eba25b9d41661ceadabb7de07f55513b37a86aa6726549131c6
+size 55532666

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98dde8a31661c3bfccfbe31571abd204d168c96dc45d745db8bcfaebc1430dd7
+size 14244

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82df9b5e5b25d86cda427dd16df6f0954def162605315f0351fa34cb0b8c6cb4
+size 1064

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606/trainer_state.json

@@ -0,0 +1,3285 @@
+{
+  "best_metric": 0.2722988724708557,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.9-num-19413-sd-10000/checkpoint-4606",
+  "epoch": 4.0,
+  "eval_steps": 10,
+  "global_step": 4606,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008684324793747287,
+      "grad_norm": 1.1732063293457031,
+      "learning_rate": 0.0002,
+      "loss": 1.7067,
+      "step": 10
+    },
+    {
+      "epoch": 0.017368649587494574,
+      "grad_norm": 1.7461568117141724,
+      "learning_rate": 0.0002,
+      "loss": 1.1596,
+      "step": 20
+    },
+    {
+      "epoch": 0.026052974381241857,
+      "grad_norm": 1.1278692483901978,
+      "learning_rate": 0.0002,
+      "loss": 0.9883,
+      "step": 30
+    },
+    {
+      "epoch": 0.03473729917498915,
+      "grad_norm": 1.2258220911026,
+      "learning_rate": 0.0002,
+      "loss": 0.9004,
+      "step": 40
+    },
+    {
+      "epoch": 0.04342162396873643,
+      "grad_norm": 1.0578727722167969,
+      "learning_rate": 0.0002,
+      "loss": 0.7675,
+      "step": 50
+    },
+    {
+      "epoch": 0.052105948762483714,
+      "grad_norm": 1.060376763343811,
+      "learning_rate": 0.0002,
+      "loss": 0.7039,
+      "step": 60
+    },
+    {
+      "epoch": 0.060790273556231005,
+      "grad_norm": 0.9346579313278198,
+      "learning_rate": 0.0002,
+      "loss": 0.7107,
+      "step": 70
+    },
+    {
+      "epoch": 0.0694745983499783,
+      "grad_norm": 0.7925026416778564,
+      "learning_rate": 0.0002,
+      "loss": 0.6864,
+      "step": 80
+    },
+    {
+      "epoch": 0.07815892314372558,
+      "grad_norm": 0.7674112319946289,
+      "learning_rate": 0.0002,
+      "loss": 0.6753,
+      "step": 90
+    },
+    {
+      "epoch": 0.08684324793747286,
+      "grad_norm": 0.8018007278442383,
+      "learning_rate": 0.0002,
+      "loss": 0.628,
+      "step": 100
+    },
+    {
+      "epoch": 0.09552757273122015,
+      "grad_norm": 0.8229703903198242,
+      "learning_rate": 0.0002,
+      "loss": 0.6552,
+      "step": 110
+    },
+    {
+      "epoch": 0.10421189752496743,
+      "grad_norm": 1.0095752477645874,
+      "learning_rate": 0.0002,
+      "loss": 0.6256,
+      "step": 120
+    },
+    {
+      "epoch": 0.11289622231871473,
+      "grad_norm": 1.26262629032135,
+      "learning_rate": 0.0002,
+      "loss": 0.6208,
+      "step": 130
+    },
+    {
+      "epoch": 0.12158054711246201,
+      "grad_norm": 0.6695842742919922,
+      "learning_rate": 0.0002,
+      "loss": 0.6167,
+      "step": 140
+    },
+    {
+      "epoch": 0.13026487190620928,
+      "grad_norm": 0.8266357779502869,
+      "learning_rate": 0.0002,
+      "loss": 0.6326,
+      "step": 150
+    },
+    {
+      "epoch": 0.1389491966999566,
+      "grad_norm": 0.9205271005630493,
+      "learning_rate": 0.0002,
+      "loss": 0.6187,
+      "step": 160
+    },
+    {
+      "epoch": 0.14763352149370387,
+      "grad_norm": 0.6115465760231018,
+      "learning_rate": 0.0002,
+      "loss": 0.6198,
+      "step": 170
+    },
+    {
+      "epoch": 0.15631784628745116,
+      "grad_norm": 0.6563206315040588,
+      "learning_rate": 0.0002,
+      "loss": 0.6065,
+      "step": 180
+    },
+    {
+      "epoch": 0.16500217108119844,
+      "grad_norm": 1.2801162004470825,
+      "learning_rate": 0.0002,
+      "loss": 0.6164,
+      "step": 190
+    },
+    {
+      "epoch": 0.17368649587494572,
+      "grad_norm": 0.5789668560028076,
+      "learning_rate": 0.0002,
+      "loss": 0.5769,
+      "step": 200
+    },
+    {
+      "epoch": 0.182370820668693,
+      "grad_norm": 0.6813567876815796,
+      "learning_rate": 0.0002,
+      "loss": 0.6087,
+      "step": 210
+    },
+    {
+      "epoch": 0.1910551454624403,
+      "grad_norm": 0.5566724538803101,
+      "learning_rate": 0.0002,
+      "loss": 0.6034,
+      "step": 220
+    },
+    {
+      "epoch": 0.19973947025618757,
+      "grad_norm": 0.7519480586051941,
+      "learning_rate": 0.0002,
+      "loss": 0.5839,
+      "step": 230
+    },
+    {
+      "epoch": 0.20842379504993486,
+      "grad_norm": 0.6408759951591492,
+      "learning_rate": 0.0002,
+      "loss": 0.5992,
+      "step": 240
+    },
+    {
+      "epoch": 0.21710811984368214,
+      "grad_norm": 0.5549838542938232,
+      "learning_rate": 0.0002,
+      "loss": 0.5965,
+      "step": 250
+    },
+    {
+      "epoch": 0.22579244463742945,
+      "grad_norm": 0.5199561715126038,
+      "learning_rate": 0.0002,
+      "loss": 0.5733,
+      "step": 260
+    },
+    {
+      "epoch": 0.23447676943117673,
+      "grad_norm": 0.6727754473686218,
+      "learning_rate": 0.0002,
+      "loss": 0.5886,
+      "step": 270
+    },
+    {
+      "epoch": 0.24316109422492402,
+      "grad_norm": 0.6133880019187927,
+      "learning_rate": 0.0002,
+      "loss": 0.5917,
+      "step": 280
+    },
+    {
+      "epoch": 0.2518454190186713,
+      "grad_norm": 0.5724732279777527,
+      "learning_rate": 0.0002,
+      "loss": 0.5565,
+      "step": 290
+    },
+    {
+      "epoch": 0.26052974381241856,
+      "grad_norm": 0.5647387504577637,
+      "learning_rate": 0.0002,
+      "loss": 0.5755,
+      "step": 300
+    },
+    {
+      "epoch": 0.26921406860616587,
+      "grad_norm": 0.5221774578094482,
+      "learning_rate": 0.0002,
+      "loss": 0.5629,
+      "step": 310
+    },
+    {
+      "epoch": 0.2778983933999132,
+      "grad_norm": 0.6939321160316467,
+      "learning_rate": 0.0002,
+      "loss": 0.5807,
+      "step": 320
+    },
+    {
+      "epoch": 0.28658271819366044,
+      "grad_norm": 0.6391360759735107,
+      "learning_rate": 0.0002,
+      "loss": 0.5584,
+      "step": 330
+    },
+    {
+      "epoch": 0.29526704298740775,
+      "grad_norm": 0.6072664260864258,
+      "learning_rate": 0.0002,
+      "loss": 0.5979,
+      "step": 340
+    },
+    {
+      "epoch": 0.303951367781155,
+      "grad_norm": 0.5596950650215149,
+      "learning_rate": 0.0002,
+      "loss": 0.5448,
+      "step": 350
+    },
+    {
+      "epoch": 0.3126356925749023,
+      "grad_norm": 0.5198812484741211,
+      "learning_rate": 0.0002,
+      "loss": 0.5436,
+      "step": 360
+    },
+    {
+      "epoch": 0.32132001736864957,
+      "grad_norm": 0.5412276387214661,
+      "learning_rate": 0.0002,
+      "loss": 0.5524,
+      "step": 370
+    },
+    {
+      "epoch": 0.3300043421623969,
+      "grad_norm": 0.6245805621147156,
+      "learning_rate": 0.0002,
+      "loss": 0.5465,
+      "step": 380
+    },
+    {
+      "epoch": 0.33868866695614414,
+      "grad_norm": 0.5097344517707825,
+      "learning_rate": 0.0002,
+      "loss": 0.5522,
+      "step": 390
+    },
+    {
+      "epoch": 0.34737299174989145,
+      "grad_norm": 0.6196363568305969,
+      "learning_rate": 0.0002,
+      "loss": 0.5478,
+      "step": 400
+    },
+    {
+      "epoch": 0.35605731654363876,
+      "grad_norm": 0.5543810129165649,
+      "learning_rate": 0.0002,
+      "loss": 0.5566,
+      "step": 410
+    },
+    {
+      "epoch": 0.364741641337386,
+      "grad_norm": 0.5921859741210938,
+      "learning_rate": 0.0002,
+      "loss": 0.5583,
+      "step": 420
+    },
+    {
+      "epoch": 0.3734259661311333,
+      "grad_norm": 0.5664304494857788,
+      "learning_rate": 0.0002,
+      "loss": 0.5439,
+      "step": 430
+    },
+    {
+      "epoch": 0.3821102909248806,
+      "grad_norm": 0.5977340340614319,
+      "learning_rate": 0.0002,
+      "loss": 0.5438,
+      "step": 440
+    },
+    {
+      "epoch": 0.3907946157186279,
+      "grad_norm": 0.6227905750274658,
+      "learning_rate": 0.0002,
+      "loss": 0.5419,
+      "step": 450
+    },
+    {
+      "epoch": 0.39947894051237515,
+      "grad_norm": 0.6239493489265442,
+      "learning_rate": 0.0002,
+      "loss": 0.5055,
+      "step": 460
+    },
+    {
+      "epoch": 0.40816326530612246,
+      "grad_norm": 0.7173473238945007,
+      "learning_rate": 0.0002,
+      "loss": 0.5436,
+      "step": 470
+    },
+    {
+      "epoch": 0.4168475900998697,
+      "grad_norm": 0.581386387348175,
+      "learning_rate": 0.0002,
+      "loss": 0.5413,
+      "step": 480
+    },
+    {
+      "epoch": 0.425531914893617,
+      "grad_norm": 0.8495563864707947,
+      "learning_rate": 0.0002,
+      "loss": 0.5313,
+      "step": 490
+    },
+    {
+      "epoch": 0.4342162396873643,
+      "grad_norm": 0.7008602023124695,
+      "learning_rate": 0.0002,
+      "loss": 0.5408,
+      "step": 500
+    },
+    {
+      "epoch": 0.4429005644811116,
+      "grad_norm": 0.7453230023384094,
+      "learning_rate": 0.0002,
+      "loss": 0.5174,
+      "step": 510
+    },
+    {
+      "epoch": 0.4515848892748589,
+      "grad_norm": 0.7419916987419128,
+      "learning_rate": 0.0002,
+      "loss": 0.5358,
+      "step": 520
+    },
+    {
+      "epoch": 0.46026921406860616,
+      "grad_norm": 0.5828579664230347,
+      "learning_rate": 0.0002,
+      "loss": 0.5382,
+      "step": 530
+    },
+    {
+      "epoch": 0.46895353886235347,
+      "grad_norm": 0.7366796135902405,
+      "learning_rate": 0.0002,
+      "loss": 0.5182,
+      "step": 540
+    },
+    {
+      "epoch": 0.4776378636561007,
+      "grad_norm": 0.5648167729377747,
+      "learning_rate": 0.0002,
+      "loss": 0.5158,
+      "step": 550
+    },
+    {
+      "epoch": 0.48632218844984804,
+      "grad_norm": 0.6356498003005981,
+      "learning_rate": 0.0002,
+      "loss": 0.5097,
+      "step": 560
+    },
+    {
+      "epoch": 0.4950065132435953,
+      "grad_norm": 0.6379111409187317,
+      "learning_rate": 0.0002,
+      "loss": 0.5235,
+      "step": 570
+    },
+    {
+      "epoch": 0.5036908380373426,
+      "grad_norm": 0.6728289723396301,
+      "learning_rate": 0.0002,
+      "loss": 0.4877,
+      "step": 580
+    },
+    {
+      "epoch": 0.5123751628310899,
+      "grad_norm": 0.6453984379768372,
+      "learning_rate": 0.0002,
+      "loss": 0.4977,
+      "step": 590
+    },
+    {
+      "epoch": 0.5210594876248371,
+      "grad_norm": 0.7061713933944702,
+      "learning_rate": 0.0002,
+      "loss": 0.5135,
+      "step": 600
+    },
+    {
+      "epoch": 0.5297438124185845,
+      "grad_norm": 0.8628040552139282,
+      "learning_rate": 0.0002,
+      "loss": 0.4973,
+      "step": 610
+    },
+    {
+      "epoch": 0.5384281372123317,
+      "grad_norm": 1.0208714008331299,
+      "learning_rate": 0.0002,
+      "loss": 0.5069,
+      "step": 620
+    },
+    {
+      "epoch": 0.547112462006079,
+      "grad_norm": 0.8920316100120544,
+      "learning_rate": 0.0002,
+      "loss": 0.4906,
+      "step": 630
+    },
+    {
+      "epoch": 0.5557967867998264,
+      "grad_norm": 0.7505720853805542,
+      "learning_rate": 0.0002,
+      "loss": 0.5252,
+      "step": 640
+    },
+    {
+      "epoch": 0.5644811115935736,
+      "grad_norm": 0.7085066437721252,
+      "learning_rate": 0.0002,
+      "loss": 0.4916,
+      "step": 650
+    },
+    {
+      "epoch": 0.5731654363873209,
+      "grad_norm": 0.9441248774528503,
+      "learning_rate": 0.0002,
+      "loss": 0.5074,
+      "step": 660
+    },
+    {
+      "epoch": 0.5818497611810681,
+      "grad_norm": 0.5366020202636719,
+      "learning_rate": 0.0002,
+      "loss": 0.4915,
+      "step": 670
+    },
+    {
+      "epoch": 0.5905340859748155,
+      "grad_norm": 0.5895682573318481,
+      "learning_rate": 0.0002,
+      "loss": 0.4824,
+      "step": 680
+    },
+    {
+      "epoch": 0.5992184107685627,
+      "grad_norm": 0.9745811820030212,
+      "learning_rate": 0.0002,
+      "loss": 0.4954,
+      "step": 690
+    },
+    {
+      "epoch": 0.60790273556231,
+      "grad_norm": 0.7926701903343201,
+      "learning_rate": 0.0002,
+      "loss": 0.4916,
+      "step": 700
+    },
+    {
+      "epoch": 0.6165870603560574,
+      "grad_norm": 0.5723416209220886,
+      "learning_rate": 0.0002,
+      "loss": 0.4691,
+      "step": 710
+    },
+    {
+      "epoch": 0.6252713851498046,
+      "grad_norm": 0.6463170051574707,
+      "learning_rate": 0.0002,
+      "loss": 0.4941,
+      "step": 720
+    },
+    {
+      "epoch": 0.6339557099435519,
+      "grad_norm": 0.6526642441749573,
+      "learning_rate": 0.0002,
+      "loss": 0.4702,
+      "step": 730
+    },
+    {
+      "epoch": 0.6426400347372991,
+      "grad_norm": 0.7835870385169983,
+      "learning_rate": 0.0002,
+      "loss": 0.4646,
+      "step": 740
+    },
+    {
+      "epoch": 0.6513243595310465,
+      "grad_norm": 0.7449156641960144,
+      "learning_rate": 0.0002,
+      "loss": 0.456,
+      "step": 750
+    },
+    {
+      "epoch": 0.6600086843247938,
+      "grad_norm": 0.6138056516647339,
+      "learning_rate": 0.0002,
+      "loss": 0.4664,
+      "step": 760
+    },
+    {
+      "epoch": 0.668693009118541,
+      "grad_norm": 0.6755684614181519,
+      "learning_rate": 0.0002,
+      "loss": 0.4743,
+      "step": 770
+    },
+    {
+      "epoch": 0.6773773339122883,
+      "grad_norm": 0.7654828429222107,
+      "learning_rate": 0.0002,
+      "loss": 0.4624,
+      "step": 780
+    },
+    {
+      "epoch": 0.6860616587060356,
+      "grad_norm": 0.8211193084716797,
+      "learning_rate": 0.0002,
+      "loss": 0.482,
+      "step": 790
+    },
+    {
+      "epoch": 0.6947459834997829,
+      "grad_norm": 0.6197503805160522,
+      "learning_rate": 0.0002,
+      "loss": 0.4783,
+      "step": 800
+    },
+    {
+      "epoch": 0.7034303082935301,
+      "grad_norm": 0.8030904531478882,
+      "learning_rate": 0.0002,
+      "loss": 0.461,
+      "step": 810
+    },
+    {
+      "epoch": 0.7121146330872775,
+      "grad_norm": 0.5908632278442383,
+      "learning_rate": 0.0002,
+      "loss": 0.4726,
+      "step": 820
+    },
+    {
+      "epoch": 0.7207989578810248,
+      "grad_norm": 0.867847204208374,
+      "learning_rate": 0.0002,
+      "loss": 0.4683,
+      "step": 830
+    },
+    {
+      "epoch": 0.729483282674772,
+      "grad_norm": 0.8335438966751099,
+      "learning_rate": 0.0002,
+      "loss": 0.4494,
+      "step": 840
+    },
+    {
+      "epoch": 0.7381676074685193,
+      "grad_norm": 0.7276930212974548,
+      "learning_rate": 0.0002,
+      "loss": 0.4446,
+      "step": 850
+    },
+    {
+      "epoch": 0.7468519322622666,
+      "grad_norm": 0.6241617202758789,
+      "learning_rate": 0.0002,
+      "loss": 0.4568,
+      "step": 860
+    },
+    {
+      "epoch": 0.7555362570560139,
+      "grad_norm": 0.6197535395622253,
+      "learning_rate": 0.0002,
+      "loss": 0.4652,
+      "step": 870
+    },
+    {
+      "epoch": 0.7642205818497612,
+      "grad_norm": 0.7276974320411682,
+      "learning_rate": 0.0002,
+      "loss": 0.4518,
+      "step": 880
+    },
+    {
+      "epoch": 0.7729049066435084,
+      "grad_norm": 0.7494757771492004,
+      "learning_rate": 0.0002,
+      "loss": 0.4533,
+      "step": 890
+    },
+    {
+      "epoch": 0.7815892314372558,
+      "grad_norm": 0.6311206817626953,
+      "learning_rate": 0.0002,
+      "loss": 0.4534,
+      "step": 900
+    },
+    {
+      "epoch": 0.790273556231003,
+      "grad_norm": 0.6753235459327698,
+      "learning_rate": 0.0002,
+      "loss": 0.4435,
+      "step": 910
+    },
+    {
+      "epoch": 0.7989578810247503,
+      "grad_norm": 0.7800608277320862,
+      "learning_rate": 0.0002,
+      "loss": 0.4602,
+      "step": 920
+    },
+    {
+      "epoch": 0.8076422058184977,
+      "grad_norm": 0.6053752899169922,
+      "learning_rate": 0.0002,
+      "loss": 0.4502,
+      "step": 930
+    },
+    {
+      "epoch": 0.8163265306122449,
+      "grad_norm": 0.8059675097465515,
+      "learning_rate": 0.0002,
+      "loss": 0.4454,
+      "step": 940
+    },
+    {
+      "epoch": 0.8250108554059922,
+      "grad_norm": 0.7138864398002625,
+      "learning_rate": 0.0002,
+      "loss": 0.4626,
+      "step": 950
+    },
+    {
+      "epoch": 0.8336951801997394,
+      "grad_norm": 0.7769238948822021,
+      "learning_rate": 0.0002,
+      "loss": 0.4327,
+      "step": 960
+    },
+    {
+      "epoch": 0.8423795049934868,
+      "grad_norm": 0.7062289118766785,
+      "learning_rate": 0.0002,
+      "loss": 0.4478,
+      "step": 970
+    },
+    {
+      "epoch": 0.851063829787234,
+      "grad_norm": 0.744469940662384,
+      "learning_rate": 0.0002,
+      "loss": 0.4329,
+      "step": 980
+    },
+    {
+      "epoch": 0.8597481545809813,
+      "grad_norm": 0.6722288727760315,
+      "learning_rate": 0.0002,
+      "loss": 0.4303,
+      "step": 990
+    },
+    {
+      "epoch": 0.8684324793747286,
+      "grad_norm": 0.7886863946914673,
+      "learning_rate": 0.0002,
+      "loss": 0.4232,
+      "step": 1000
+    },
+    {
+      "epoch": 0.8771168041684759,
+      "grad_norm": 0.8501645922660828,
+      "learning_rate": 0.0002,
+      "loss": 0.4201,
+      "step": 1010
+    },
+    {
+      "epoch": 0.8858011289622232,
+      "grad_norm": 0.9484928250312805,
+      "learning_rate": 0.0002,
+      "loss": 0.4491,
+      "step": 1020
+    },
+    {
+      "epoch": 0.8944854537559704,
+      "grad_norm": 0.6376281976699829,
+      "learning_rate": 0.0002,
+      "loss": 0.4121,
+      "step": 1030
+    },
+    {
+      "epoch": 0.9031697785497178,
+      "grad_norm": 0.5941802263259888,
+      "learning_rate": 0.0002,
+      "loss": 0.4223,
+      "step": 1040
+    },
+    {
+      "epoch": 0.9118541033434651,
+      "grad_norm": 0.75499027967453,
+      "learning_rate": 0.0002,
+      "loss": 0.4319,
+      "step": 1050
+    },
+    {
+      "epoch": 0.9205384281372123,
+      "grad_norm": 0.8150404095649719,
+      "learning_rate": 0.0002,
+      "loss": 0.4286,
+      "step": 1060
+    },
+    {
+      "epoch": 0.9292227529309596,
+      "grad_norm": 0.8157607316970825,
+      "learning_rate": 0.0002,
+      "loss": 0.4379,
+      "step": 1070
+    },
+    {
+      "epoch": 0.9379070777247069,
+      "grad_norm": 0.6417438387870789,
+      "learning_rate": 0.0002,
+      "loss": 0.4459,
+      "step": 1080
+    },
+    {
+      "epoch": 0.9465914025184542,
+      "grad_norm": 0.8608890175819397,
+      "learning_rate": 0.0002,
+      "loss": 0.4161,
+      "step": 1090
+    },
+    {
+      "epoch": 0.9552757273122015,
+      "grad_norm": 0.5620037913322449,
+      "learning_rate": 0.0002,
+      "loss": 0.4155,
+      "step": 1100
+    },
+    {
+      "epoch": 0.9639600521059488,
+      "grad_norm": 1.3118562698364258,
+      "learning_rate": 0.0002,
+      "loss": 0.4215,
+      "step": 1110
+    },
+    {
+      "epoch": 0.9726443768996961,
+      "grad_norm": 0.8373771905899048,
+      "learning_rate": 0.0002,
+      "loss": 0.4017,
+      "step": 1120
+    },
+    {
+      "epoch": 0.9813287016934433,
+      "grad_norm": 0.7725268602371216,
+      "learning_rate": 0.0002,
+      "loss": 0.4014,
+      "step": 1130
+    },
+    {
+      "epoch": 0.9900130264871906,
+      "grad_norm": 0.7564753890037537,
+      "learning_rate": 0.0002,
+      "loss": 0.4143,
+      "step": 1140
+    },
+    {
+      "epoch": 0.998697351280938,
+      "grad_norm": 0.7307513952255249,
+      "learning_rate": 0.0002,
+      "loss": 0.4085,
+      "step": 1150
+    },
+    {
+      "epoch": 0.9995657837603127,
+      "eval_loss": 0.4137004315853119,
+      "eval_runtime": 51.9598,
+      "eval_samples_per_second": 7.025,
+      "eval_steps_per_second": 0.885,
+      "step": 1151
+    },
+    {
+      "epoch": 1.0073816760746852,
+      "grad_norm": 0.880082905292511,
+      "learning_rate": 0.0002,
+      "loss": 0.3848,
+      "step": 1160
+    },
+    {
+      "epoch": 1.0160660008684326,
+      "grad_norm": 0.7512372732162476,
+      "learning_rate": 0.0002,
+      "loss": 0.3722,
+      "step": 1170
+    },
+    {
+      "epoch": 1.0247503256621797,
+      "grad_norm": 0.6832528710365295,
+      "learning_rate": 0.0002,
+      "loss": 0.3756,
+      "step": 1180
+    },
+    {
+      "epoch": 1.033434650455927,
+      "grad_norm": 0.9188297986984253,
+      "learning_rate": 0.0002,
+      "loss": 0.3798,
+      "step": 1190
+    },
+    {
+      "epoch": 1.0421189752496742,
+      "grad_norm": 0.6144621968269348,
+      "learning_rate": 0.0002,
+      "loss": 0.3837,
+      "step": 1200
+    },
+    {
+      "epoch": 1.0508033000434216,
+      "grad_norm": 0.750042200088501,
+      "learning_rate": 0.0002,
+      "loss": 0.3893,
+      "step": 1210
+    },
+    {
+      "epoch": 1.059487624837169,
+      "grad_norm": 0.6084040999412537,
+      "learning_rate": 0.0002,
+      "loss": 0.3793,
+      "step": 1220
+    },
+    {
+      "epoch": 1.068171949630916,
+      "grad_norm": 0.8569493889808655,
+      "learning_rate": 0.0002,
+      "loss": 0.3857,
+      "step": 1230
+    },
+    {
+      "epoch": 1.0768562744246635,
+      "grad_norm": 0.8420213460922241,
+      "learning_rate": 0.0002,
+      "loss": 0.381,
+      "step": 1240
+    },
+    {
+      "epoch": 1.0855405992184108,
+      "grad_norm": 0.5716078281402588,
+      "learning_rate": 0.0002,
+      "loss": 0.3905,
+      "step": 1250
+    },
+    {
+      "epoch": 1.094224924012158,
+      "grad_norm": 0.6244675517082214,
+      "learning_rate": 0.0002,
+      "loss": 0.3578,
+      "step": 1260
+    },
+    {
+      "epoch": 1.1029092488059054,
+      "grad_norm": 0.7346081733703613,
+      "learning_rate": 0.0002,
+      "loss": 0.3817,
+      "step": 1270
+    },
+    {
+      "epoch": 1.1115935735996527,
+      "grad_norm": 1.0054655075073242,
+      "learning_rate": 0.0002,
+      "loss": 0.3738,
+      "step": 1280
+    },
+    {
+      "epoch": 1.1202778983933999,
+      "grad_norm": 0.7442952394485474,
+      "learning_rate": 0.0002,
+      "loss": 0.3778,
+      "step": 1290
+    },
+    {
+      "epoch": 1.1289622231871472,
+      "grad_norm": 0.7852597236633301,
+      "learning_rate": 0.0002,
+      "loss": 0.3996,
+      "step": 1300
+    },
+    {
+      "epoch": 1.1376465479808946,
+      "grad_norm": 0.9616802334785461,
+      "learning_rate": 0.0002,
+      "loss": 0.3878,
+      "step": 1310
+    },
+    {
+      "epoch": 1.1463308727746417,
+      "grad_norm": 0.7134007811546326,
+      "learning_rate": 0.0002,
+      "loss": 0.3747,
+      "step": 1320
+    },
+    {
+      "epoch": 1.155015197568389,
+      "grad_norm": 0.6189036965370178,
+      "learning_rate": 0.0002,
+      "loss": 0.3609,
+      "step": 1330
+    },
+    {
+      "epoch": 1.1636995223621363,
+      "grad_norm": 0.7725108861923218,
+      "learning_rate": 0.0002,
+      "loss": 0.3793,
+      "step": 1340
+    },
+    {
+      "epoch": 1.1723838471558836,
+      "grad_norm": 0.8503804802894592,
+      "learning_rate": 0.0002,
+      "loss": 0.3687,
+      "step": 1350
+    },
+    {
+      "epoch": 1.181068171949631,
+      "grad_norm": 0.7709351181983948,
+      "learning_rate": 0.0002,
+      "loss": 0.3767,
+      "step": 1360
+    },
+    {
+      "epoch": 1.1897524967433781,
+      "grad_norm": 0.9070065021514893,
+      "learning_rate": 0.0002,
+      "loss": 0.3914,
+      "step": 1370
+    },
+    {
+      "epoch": 1.1984368215371255,
+      "grad_norm": 0.7514732480049133,
+      "learning_rate": 0.0002,
+      "loss": 0.3962,
+      "step": 1380
+    },
+    {
+      "epoch": 1.2071211463308729,
+      "grad_norm": 0.9331297874450684,
+      "learning_rate": 0.0002,
+      "loss": 0.3597,
+      "step": 1390
+    },
+    {
+      "epoch": 1.21580547112462,
+      "grad_norm": 0.9618472456932068,
+      "learning_rate": 0.0002,
+      "loss": 0.3764,
+      "step": 1400
+    },
+    {
+      "epoch": 1.2244897959183674,
+      "grad_norm": 0.8956435918807983,
+      "learning_rate": 0.0002,
+      "loss": 0.3709,
+      "step": 1410
+    },
+    {
+      "epoch": 1.2331741207121145,
+      "grad_norm": 0.9150217175483704,
+      "learning_rate": 0.0002,
+      "loss": 0.3775,
+      "step": 1420
+    },
+    {
+      "epoch": 1.2418584455058619,
+      "grad_norm": 0.6232621073722839,
+      "learning_rate": 0.0002,
+      "loss": 0.3656,
+      "step": 1430
+    },
+    {
+      "epoch": 1.2505427702996093,
+      "grad_norm": 0.980364978313446,
+      "learning_rate": 0.0002,
+      "loss": 0.3831,
+      "step": 1440
+    },
+    {
+      "epoch": 1.2592270950933564,
+      "grad_norm": 0.8638973236083984,
+      "learning_rate": 0.0002,
+      "loss": 0.3514,
+      "step": 1450
+    },
+    {
+      "epoch": 1.2679114198871038,
+      "grad_norm": 0.6402583122253418,
+      "learning_rate": 0.0002,
+      "loss": 0.3659,
+      "step": 1460
+    },
+    {
+      "epoch": 1.2765957446808511,
+      "grad_norm": 0.9318592548370361,
+      "learning_rate": 0.0002,
+      "loss": 0.3772,
+      "step": 1470
+    },
+    {
+      "epoch": 1.2852800694745983,
+      "grad_norm": 0.7031506299972534,
+      "learning_rate": 0.0002,
+      "loss": 0.3727,
+      "step": 1480
+    },
+    {
+      "epoch": 1.2939643942683456,
+      "grad_norm": 0.9225306510925293,
+      "learning_rate": 0.0002,
+      "loss": 0.3615,
+      "step": 1490
+    },
+    {
+      "epoch": 1.302648719062093,
+      "grad_norm": 0.8340631127357483,
+      "learning_rate": 0.0002,
+      "loss": 0.3838,
+      "step": 1500
+    },
+    {
+      "epoch": 1.3113330438558402,
+      "grad_norm": 0.9431206583976746,
+      "learning_rate": 0.0002,
+      "loss": 0.3747,
+      "step": 1510
+    },
+    {
+      "epoch": 1.3200173686495875,
+      "grad_norm": 0.8804561495780945,
+      "learning_rate": 0.0002,
+      "loss": 0.3535,
+      "step": 1520
+    },
+    {
+      "epoch": 1.3287016934433349,
+      "grad_norm": 0.8626439571380615,
+      "learning_rate": 0.0002,
+      "loss": 0.3679,
+      "step": 1530
+    },
+    {
+      "epoch": 1.337386018237082,
+      "grad_norm": 0.8414568305015564,
+      "learning_rate": 0.0002,
+      "loss": 0.3644,
+      "step": 1540
+    },
+    {
+      "epoch": 1.3460703430308294,
+      "grad_norm": 1.127997636795044,
+      "learning_rate": 0.0002,
+      "loss": 0.3604,
+      "step": 1550
+    },
+    {
+      "epoch": 1.3547546678245768,
+      "grad_norm": 0.7714207172393799,
+      "learning_rate": 0.0002,
+      "loss": 0.3589,
+      "step": 1560
+    },
+    {
+      "epoch": 1.363438992618324,
+      "grad_norm": 0.9359990954399109,
+      "learning_rate": 0.0002,
+      "loss": 0.3661,
+      "step": 1570
+    },
+    {
+      "epoch": 1.3721233174120713,
+      "grad_norm": 0.7754091620445251,
+      "learning_rate": 0.0002,
+      "loss": 0.3694,
+      "step": 1580
+    },
+    {
+      "epoch": 1.3808076422058184,
+      "grad_norm": 1.0162203311920166,
+      "learning_rate": 0.0002,
+      "loss": 0.36,
+      "step": 1590
+    },
+    {
+      "epoch": 1.3894919669995658,
+      "grad_norm": 0.9444917440414429,
+      "learning_rate": 0.0002,
+      "loss": 0.347,
+      "step": 1600
+    },
+    {
+      "epoch": 1.3981762917933132,
+      "grad_norm": 0.8122510313987732,
+      "learning_rate": 0.0002,
+      "loss": 0.3625,
+      "step": 1610
+    },
+    {
+      "epoch": 1.4068606165870603,
+      "grad_norm": 0.7946245670318604,
+      "learning_rate": 0.0002,
+      "loss": 0.3499,
+      "step": 1620
+    },
+    {
+      "epoch": 1.4155449413808077,
+      "grad_norm": 0.899692714214325,
+      "learning_rate": 0.0002,
+      "loss": 0.3606,
+      "step": 1630
+    },
+    {
+      "epoch": 1.4242292661745548,
+      "grad_norm": 0.652953565120697,
+      "learning_rate": 0.0002,
+      "loss": 0.3599,
+      "step": 1640
+    },
+    {
+      "epoch": 1.4329135909683022,
+      "grad_norm": 0.6942960023880005,
+      "learning_rate": 0.0002,
+      "loss": 0.3585,
+      "step": 1650
+    },
+    {
+      "epoch": 1.4415979157620495,
+      "grad_norm": 0.5737764239311218,
+      "learning_rate": 0.0002,
+      "loss": 0.3624,
+      "step": 1660
+    },
+    {
+      "epoch": 1.4502822405557967,
+      "grad_norm": 0.7966623306274414,
+      "learning_rate": 0.0002,
+      "loss": 0.3446,
+      "step": 1670
+    },
+    {
+      "epoch": 1.458966565349544,
+      "grad_norm": 0.7757281064987183,
+      "learning_rate": 0.0002,
+      "loss": 0.34,
+      "step": 1680
+    },
+    {
+      "epoch": 1.4676508901432914,
+      "grad_norm": 0.7565333843231201,
+      "learning_rate": 0.0002,
+      "loss": 0.377,
+      "step": 1690
+    },
+    {
+      "epoch": 1.4763352149370386,
+      "grad_norm": 1.2639003992080688,
+      "learning_rate": 0.0002,
+      "loss": 0.3536,
+      "step": 1700
+    },
+    {
+      "epoch": 1.485019539730786,
+      "grad_norm": 1.0366960763931274,
+      "learning_rate": 0.0002,
+      "loss": 0.3503,
+      "step": 1710
+    },
+    {
+      "epoch": 1.4937038645245333,
+      "grad_norm": 0.8083975911140442,
+      "learning_rate": 0.0002,
+      "loss": 0.3594,
+      "step": 1720
+    },
+    {
+      "epoch": 1.5023881893182804,
+      "grad_norm": 1.0527535676956177,
+      "learning_rate": 0.0002,
+      "loss": 0.369,
+      "step": 1730
+    },
+    {
+      "epoch": 1.5110725141120278,
+      "grad_norm": 0.7138310074806213,
+      "learning_rate": 0.0002,
+      "loss": 0.3424,
+      "step": 1740
+    },
+    {
+      "epoch": 1.5197568389057752,
+      "grad_norm": 0.818398118019104,
+      "learning_rate": 0.0002,
+      "loss": 0.3568,
+      "step": 1750
+    },
+    {
+      "epoch": 1.5284411636995223,
+      "grad_norm": 0.7518394589424133,
+      "learning_rate": 0.0002,
+      "loss": 0.3347,
+      "step": 1760
+    },
+    {
+      "epoch": 1.5371254884932697,
+      "grad_norm": 0.6814934611320496,
+      "learning_rate": 0.0002,
+      "loss": 0.3462,
+      "step": 1770
+    },
+    {
+      "epoch": 1.545809813287017,
+      "grad_norm": 0.49772709608078003,
+      "learning_rate": 0.0002,
+      "loss": 0.3432,
+      "step": 1780
+    },
+    {
+      "epoch": 1.5544941380807642,
+      "grad_norm": 0.7579216957092285,
+      "learning_rate": 0.0002,
+      "loss": 0.3457,
+      "step": 1790
+    },
+    {
+      "epoch": 1.5631784628745113,
+      "grad_norm": 0.7244242429733276,
+      "learning_rate": 0.0002,
+      "loss": 0.3517,
+      "step": 1800
+    },
+    {
+      "epoch": 1.571862787668259,
+      "grad_norm": 0.6358800530433655,
+      "learning_rate": 0.0002,
+      "loss": 0.3427,
+      "step": 1810
+    },
+    {
+      "epoch": 1.580547112462006,
+      "grad_norm": 0.6636850833892822,
+      "learning_rate": 0.0002,
+      "loss": 0.3482,
+      "step": 1820
+    },
+    {
+      "epoch": 1.5892314372557532,
+      "grad_norm": 0.691331148147583,
+      "learning_rate": 0.0002,
+      "loss": 0.3397,
+      "step": 1830
+    },
+    {
+      "epoch": 1.5979157620495008,
+      "grad_norm": 0.8501580357551575,
+      "learning_rate": 0.0002,
+      "loss": 0.3525,
+      "step": 1840
+    },
+    {
+      "epoch": 1.606600086843248,
+      "grad_norm": 0.8448147177696228,
+      "learning_rate": 0.0002,
+      "loss": 0.3319,
+      "step": 1850
+    },
+    {
+      "epoch": 1.615284411636995,
+      "grad_norm": 0.6357255578041077,
+      "learning_rate": 0.0002,
+      "loss": 0.3355,
+      "step": 1860
+    },
+    {
+      "epoch": 1.6239687364307425,
+      "grad_norm": 1.008238673210144,
+      "learning_rate": 0.0002,
+      "loss": 0.3592,
+      "step": 1870
+    },
+    {
+      "epoch": 1.6326530612244898,
+      "grad_norm": 0.7839747667312622,
+      "learning_rate": 0.0002,
+      "loss": 0.3342,
+      "step": 1880
+    },
+    {
+      "epoch": 1.641337386018237,
+      "grad_norm": 1.1214498281478882,
+      "learning_rate": 0.0002,
+      "loss": 0.3348,
+      "step": 1890
+    },
+    {
+      "epoch": 1.6500217108119843,
+      "grad_norm": 0.9457558989524841,
+      "learning_rate": 0.0002,
+      "loss": 0.335,
+      "step": 1900
+    },
+    {
+      "epoch": 1.6587060356057317,
+      "grad_norm": 0.8412042260169983,
+      "learning_rate": 0.0002,
+      "loss": 0.3425,
+      "step": 1910
+    },
+    {
+      "epoch": 1.6673903603994789,
+      "grad_norm": 1.0219208002090454,
+      "learning_rate": 0.0002,
+      "loss": 0.3427,
+      "step": 1920
+    },
+    {
+      "epoch": 1.6760746851932262,
+      "grad_norm": 1.079763412475586,
+      "learning_rate": 0.0002,
+      "loss": 0.3429,
+      "step": 1930
+    },
+    {
+      "epoch": 1.6847590099869736,
+      "grad_norm": 0.6026722192764282,
+      "learning_rate": 0.0002,
+      "loss": 0.3433,
+      "step": 1940
+    },
+    {
+      "epoch": 1.6934433347807207,
+      "grad_norm": 0.840270459651947,
+      "learning_rate": 0.0002,
+      "loss": 0.3438,
+      "step": 1950
+    },
+    {
+      "epoch": 1.702127659574468,
+      "grad_norm": 0.8331791758537292,
+      "learning_rate": 0.0002,
+      "loss": 0.3544,
+      "step": 1960
+    },
+    {
+      "epoch": 1.7108119843682155,
+      "grad_norm": 0.6232297420501709,
+      "learning_rate": 0.0002,
+      "loss": 0.3396,
+      "step": 1970
+    },
+    {
+      "epoch": 1.7194963091619626,
+      "grad_norm": 0.8575964570045471,
+      "learning_rate": 0.0002,
+      "loss": 0.3413,
+      "step": 1980
+    },
+    {
+      "epoch": 1.72818063395571,
+      "grad_norm": 1.4170857667922974,
+      "learning_rate": 0.0002,
+      "loss": 0.3375,
+      "step": 1990
+    },
+    {
+      "epoch": 1.7368649587494573,
+      "grad_norm": 0.7481379508972168,
+      "learning_rate": 0.0002,
+      "loss": 0.3328,
+      "step": 2000
+    },
+    {
+      "epoch": 1.7455492835432045,
+      "grad_norm": 0.6883425712585449,
+      "learning_rate": 0.0002,
+      "loss": 0.3413,
+      "step": 2010
+    },
+    {
+      "epoch": 1.7542336083369519,
+      "grad_norm": 1.1543880701065063,
+      "learning_rate": 0.0002,
+      "loss": 0.3381,
+      "step": 2020
+    },
+    {
+      "epoch": 1.7629179331306992,
+      "grad_norm": 0.8788072466850281,
+      "learning_rate": 0.0002,
+      "loss": 0.3369,
+      "step": 2030
+    },
+    {
+      "epoch": 1.7716022579244464,
+      "grad_norm": 0.7034786343574524,
+      "learning_rate": 0.0002,
+      "loss": 0.3266,
+      "step": 2040
+    },
+    {
+      "epoch": 1.7802865827181935,
+      "grad_norm": 1.041170358657837,
+      "learning_rate": 0.0002,
+      "loss": 0.3263,
+      "step": 2050
+    },
+    {
+      "epoch": 1.788970907511941,
+      "grad_norm": 0.7966195344924927,
+      "learning_rate": 0.0002,
+      "loss": 0.3309,
+      "step": 2060
+    },
+    {
+      "epoch": 1.7976552323056882,
+      "grad_norm": 0.8562559485435486,
+      "learning_rate": 0.0002,
+      "loss": 0.3174,
+      "step": 2070
+    },
+    {
+      "epoch": 1.8063395570994354,
+      "grad_norm": 0.9997917413711548,
+      "learning_rate": 0.0002,
+      "loss": 0.3384,
+      "step": 2080
+    },
+    {
+      "epoch": 1.8150238818931828,
+      "grad_norm": 0.9915640950202942,
+      "learning_rate": 0.0002,
+      "loss": 0.3291,
+      "step": 2090
+    },
+    {
+      "epoch": 1.8237082066869301,
+      "grad_norm": 0.8850535154342651,
+      "learning_rate": 0.0002,
+      "loss": 0.3266,
+      "step": 2100
+    },
+    {
+      "epoch": 1.8323925314806773,
+      "grad_norm": 0.7072052359580994,
+      "learning_rate": 0.0002,
+      "loss": 0.3269,
+      "step": 2110
+    },
+    {
+      "epoch": 1.8410768562744246,
+      "grad_norm": 0.8423166871070862,
+      "learning_rate": 0.0002,
+      "loss": 0.3302,
+      "step": 2120
+    },
+    {
+      "epoch": 1.849761181068172,
+      "grad_norm": 0.7291885614395142,
+      "learning_rate": 0.0002,
+      "loss": 0.3382,
+      "step": 2130
+    },
+    {
+      "epoch": 1.8584455058619191,
+      "grad_norm": 1.0388280153274536,
+      "learning_rate": 0.0002,
+      "loss": 0.3247,
+      "step": 2140
+    },
+    {
+      "epoch": 1.8671298306556665,
+      "grad_norm": 0.8792893886566162,
+      "learning_rate": 0.0002,
+      "loss": 0.3223,
+      "step": 2150
+    },
+    {
+      "epoch": 1.8758141554494139,
+      "grad_norm": 0.852391242980957,
+      "learning_rate": 0.0002,
+      "loss": 0.3385,
+      "step": 2160
+    },
+    {
+      "epoch": 1.884498480243161,
+      "grad_norm": 0.7536265850067139,
+      "learning_rate": 0.0002,
+      "loss": 0.3253,
+      "step": 2170
+    },
+    {
+      "epoch": 1.8931828050369084,
+      "grad_norm": 0.8531852960586548,
+      "learning_rate": 0.0002,
+      "loss": 0.3234,
+      "step": 2180
+    },
+    {
+      "epoch": 1.9018671298306558,
+      "grad_norm": 0.6752887964248657,
+      "learning_rate": 0.0002,
+      "loss": 0.3352,
+      "step": 2190
+    },
+    {
+      "epoch": 1.910551454624403,
+      "grad_norm": 0.9746645092964172,
+      "learning_rate": 0.0002,
+      "loss": 0.3344,
+      "step": 2200
+    },
+    {
+      "epoch": 1.9192357794181503,
+      "grad_norm": 0.8092345595359802,
+      "learning_rate": 0.0002,
+      "loss": 0.331,
+      "step": 2210
+    },
+    {
+      "epoch": 1.9279201042118976,
+      "grad_norm": 0.6521363854408264,
+      "learning_rate": 0.0002,
+      "loss": 0.3245,
+      "step": 2220
+    },
+    {
+      "epoch": 1.9366044290056448,
+      "grad_norm": 1.0131723880767822,
+      "learning_rate": 0.0002,
+      "loss": 0.3272,
+      "step": 2230
+    },
+    {
+      "epoch": 1.9452887537993921,
+      "grad_norm": 0.6056963205337524,
+      "learning_rate": 0.0002,
+      "loss": 0.3166,
+      "step": 2240
+    },
+    {
+      "epoch": 1.9539730785931395,
+      "grad_norm": 0.9994075894355774,
+      "learning_rate": 0.0002,
+      "loss": 0.3346,
+      "step": 2250
+    },
+    {
+      "epoch": 1.9626574033868867,
+      "grad_norm": 0.8796638250350952,
+      "learning_rate": 0.0002,
+      "loss": 0.3214,
+      "step": 2260
+    },
+    {
+      "epoch": 1.9713417281806338,
+      "grad_norm": 0.6505881547927856,
+      "learning_rate": 0.0002,
+      "loss": 0.3225,
+      "step": 2270
+    },
+    {
+      "epoch": 1.9800260529743814,
+      "grad_norm": 1.1708468198776245,
+      "learning_rate": 0.0002,
+      "loss": 0.3346,
+      "step": 2280
+    },
+    {
+      "epoch": 1.9887103777681285,
+      "grad_norm": 0.898364245891571,
+      "learning_rate": 0.0002,
+      "loss": 0.3234,
+      "step": 2290
+    },
+    {
+      "epoch": 1.9973947025618757,
+      "grad_norm": 0.7017287611961365,
+      "learning_rate": 0.0002,
+      "loss": 0.3263,
+      "step": 2300
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.3223828971385956,
+      "eval_runtime": 51.2747,
+      "eval_samples_per_second": 7.119,
+      "eval_steps_per_second": 0.897,
+      "step": 2303
+    },
+    {
+      "epoch": 2.0060790273556233,
+      "grad_norm": 0.807558000087738,
+      "learning_rate": 0.0002,
+      "loss": 0.3249,
+      "step": 2310
+    },
+    {
+      "epoch": 2.0147633521493704,
+      "grad_norm": 1.107229471206665,
+      "learning_rate": 0.0002,
+      "loss": 0.3036,
+      "step": 2320
+    },
+    {
+      "epoch": 2.0234476769431176,
+      "grad_norm": 0.8265573978424072,
+      "learning_rate": 0.0002,
+      "loss": 0.3057,
+      "step": 2330
+    },
+    {
+      "epoch": 2.032132001736865,
+      "grad_norm": 0.6224291324615479,
+      "learning_rate": 0.0002,
+      "loss": 0.3073,
+      "step": 2340
+    },
+    {
+      "epoch": 2.0408163265306123,
+      "grad_norm": 0.8967212438583374,
+      "learning_rate": 0.0002,
+      "loss": 0.3086,
+      "step": 2350
+    },
+    {
+      "epoch": 2.0495006513243594,
+      "grad_norm": 0.6476389765739441,
+      "learning_rate": 0.0002,
+      "loss": 0.3059,
+      "step": 2360
+    },
+    {
+      "epoch": 2.058184976118107,
+      "grad_norm": 0.7838577628135681,
+      "learning_rate": 0.0002,
+      "loss": 0.3013,
+      "step": 2370
+    },
+    {
+      "epoch": 2.066869300911854,
+      "grad_norm": 0.9126287698745728,
+      "learning_rate": 0.0002,
+      "loss": 0.3136,
+      "step": 2380
+    },
+    {
+      "epoch": 2.0755536257056013,
+      "grad_norm": 0.946692705154419,
+      "learning_rate": 0.0002,
+      "loss": 0.3007,
+      "step": 2390
+    },
+    {
+      "epoch": 2.0842379504993485,
+      "grad_norm": 0.6643753051757812,
+      "learning_rate": 0.0002,
+      "loss": 0.2997,
+      "step": 2400
+    },
+    {
+      "epoch": 2.092922275293096,
+      "grad_norm": 0.7992640733718872,
+      "learning_rate": 0.0002,
+      "loss": 0.3077,
+      "step": 2410
+    },
+    {
+      "epoch": 2.101606600086843,
+      "grad_norm": 1.0821648836135864,
+      "learning_rate": 0.0002,
+      "loss": 0.3142,
+      "step": 2420
+    },
+    {
+      "epoch": 2.1102909248805903,
+      "grad_norm": 0.7750299572944641,
+      "learning_rate": 0.0002,
+      "loss": 0.3019,
+      "step": 2430
+    },
+    {
+      "epoch": 2.118975249674338,
+      "grad_norm": 0.8424153327941895,
+      "learning_rate": 0.0002,
+      "loss": 0.3059,
+      "step": 2440
+    },
+    {
+      "epoch": 2.127659574468085,
+      "grad_norm": 1.0579770803451538,
+      "learning_rate": 0.0002,
+      "loss": 0.301,
+      "step": 2450
+    },
+    {
+      "epoch": 2.136343899261832,
+      "grad_norm": 0.7864382863044739,
+      "learning_rate": 0.0002,
+      "loss": 0.3061,
+      "step": 2460
+    },
+    {
+      "epoch": 2.14502822405558,
+      "grad_norm": 0.8209099173545837,
+      "learning_rate": 0.0002,
+      "loss": 0.3055,
+      "step": 2470
+    },
+    {
+      "epoch": 2.153712548849327,
+      "grad_norm": 0.7030050754547119,
+      "learning_rate": 0.0002,
+      "loss": 0.2991,
+      "step": 2480
+    },
+    {
+      "epoch": 2.162396873643074,
+      "grad_norm": 0.8806471824645996,
+      "learning_rate": 0.0002,
+      "loss": 0.3212,
+      "step": 2490
+    },
+    {
+      "epoch": 2.1710811984368217,
+      "grad_norm": 0.7312551736831665,
+      "learning_rate": 0.0002,
+      "loss": 0.2991,
+      "step": 2500
+    },
+    {
+      "epoch": 2.179765523230569,
+      "grad_norm": 0.9340773224830627,
+      "learning_rate": 0.0002,
+      "loss": 0.3078,
+      "step": 2510
+    },
+    {
+      "epoch": 2.188449848024316,
+      "grad_norm": 0.633319079875946,
+      "learning_rate": 0.0002,
+      "loss": 0.3021,
+      "step": 2520
+    },
+    {
+      "epoch": 2.1971341728180636,
+      "grad_norm": 0.8805091381072998,
+      "learning_rate": 0.0002,
+      "loss": 0.3071,
+      "step": 2530
+    },
+    {
+      "epoch": 2.2058184976118107,
+      "grad_norm": 0.8670169115066528,
+      "learning_rate": 0.0002,
+      "loss": 0.2959,
+      "step": 2540
+    },
+    {
+      "epoch": 2.214502822405558,
+      "grad_norm": 0.8338320851325989,
+      "learning_rate": 0.0002,
+      "loss": 0.3025,
+      "step": 2550
+    },
+    {
+      "epoch": 2.2231871471993054,
+      "grad_norm": 0.9541687369346619,
+      "learning_rate": 0.0002,
+      "loss": 0.3024,
+      "step": 2560
+    },
+    {
+      "epoch": 2.2318714719930526,
+      "grad_norm": 0.976690948009491,
+      "learning_rate": 0.0002,
+      "loss": 0.3174,
+      "step": 2570
+    },
+    {
+      "epoch": 2.2405557967867997,
+      "grad_norm": 0.5785702466964722,
+      "learning_rate": 0.0002,
+      "loss": 0.3036,
+      "step": 2580
+    },
+    {
+      "epoch": 2.2492401215805473,
+      "grad_norm": 0.7772279381752014,
+      "learning_rate": 0.0002,
+      "loss": 0.3054,
+      "step": 2590
+    },
+    {
+      "epoch": 2.2579244463742945,
+      "grad_norm": 0.7839940786361694,
+      "learning_rate": 0.0002,
+      "loss": 0.3062,
+      "step": 2600
+    },
+    {
+      "epoch": 2.2666087711680416,
+      "grad_norm": 0.8164472579956055,
+      "learning_rate": 0.0002,
+      "loss": 0.2952,
+      "step": 2610
+    },
+    {
+      "epoch": 2.275293095961789,
+      "grad_norm": 0.9164624214172363,
+      "learning_rate": 0.0002,
+      "loss": 0.3078,
+      "step": 2620
+    },
+    {
+      "epoch": 2.2839774207555363,
+      "grad_norm": 1.0616720914840698,
+      "learning_rate": 0.0002,
+      "loss": 0.309,
+      "step": 2630
+    },
+    {
+      "epoch": 2.2926617455492835,
+      "grad_norm": 0.6295320391654968,
+      "learning_rate": 0.0002,
+      "loss": 0.307,
+      "step": 2640
+    },
+    {
+      "epoch": 2.3013460703430306,
+      "grad_norm": 0.6153804063796997,
+      "learning_rate": 0.0002,
+      "loss": 0.3003,
+      "step": 2650
+    },
+    {
+      "epoch": 2.310030395136778,
+      "grad_norm": 0.6307553648948669,
+      "learning_rate": 0.0002,
+      "loss": 0.3054,
+      "step": 2660
+    },
+    {
+      "epoch": 2.3187147199305254,
+      "grad_norm": 0.7876275777816772,
+      "learning_rate": 0.0002,
+      "loss": 0.2996,
+      "step": 2670
+    },
+    {
+      "epoch": 2.3273990447242725,
+      "grad_norm": 1.0939222574234009,
+      "learning_rate": 0.0002,
+      "loss": 0.3039,
+      "step": 2680
+    },
+    {
+      "epoch": 2.33608336951802,
+      "grad_norm": 0.755632758140564,
+      "learning_rate": 0.0002,
+      "loss": 0.3038,
+      "step": 2690
+    },
+    {
+      "epoch": 2.3447676943117672,
+      "grad_norm": 1.750001311302185,
+      "learning_rate": 0.0002,
+      "loss": 0.3009,
+      "step": 2700
+    },
+    {
+      "epoch": 2.3534520191055144,
+      "grad_norm": 0.6993144750595093,
+      "learning_rate": 0.0002,
+      "loss": 0.3077,
+      "step": 2710
+    },
+    {
+      "epoch": 2.362136343899262,
+      "grad_norm": 0.8878545165061951,
+      "learning_rate": 0.0002,
+      "loss": 0.2952,
+      "step": 2720
+    },
+    {
+      "epoch": 2.370820668693009,
+      "grad_norm": 0.737571656703949,
+      "learning_rate": 0.0002,
+      "loss": 0.3074,
+      "step": 2730
+    },
+    {
+      "epoch": 2.3795049934867563,
+      "grad_norm": 0.5662615299224854,
+      "learning_rate": 0.0002,
+      "loss": 0.2937,
+      "step": 2740
+    },
+    {
+      "epoch": 2.388189318280504,
+      "grad_norm": 1.082109808921814,
+      "learning_rate": 0.0002,
+      "loss": 0.2977,
+      "step": 2750
+    },
+    {
+      "epoch": 2.396873643074251,
+      "grad_norm": 0.761906623840332,
+      "learning_rate": 0.0002,
+      "loss": 0.2955,
+      "step": 2760
+    },
+    {
+      "epoch": 2.405557967867998,
+      "grad_norm": 1.020167350769043,
+      "learning_rate": 0.0002,
+      "loss": 0.3001,
+      "step": 2770
+    },
+    {
+      "epoch": 2.4142422926617457,
+      "grad_norm": 1.0099780559539795,
+      "learning_rate": 0.0002,
+      "loss": 0.3017,
+      "step": 2780
+    },
+    {
+      "epoch": 2.422926617455493,
+      "grad_norm": 1.0733739137649536,
+      "learning_rate": 0.0002,
+      "loss": 0.2991,
+      "step": 2790
+    },
+    {
+      "epoch": 2.43161094224924,
+      "grad_norm": 0.7780752778053284,
+      "learning_rate": 0.0002,
+      "loss": 0.309,
+      "step": 2800
+    },
+    {
+      "epoch": 2.4402952670429876,
+      "grad_norm": 0.8345295786857605,
+      "learning_rate": 0.0002,
+      "loss": 0.2924,
+      "step": 2810
+    },
+    {
+      "epoch": 2.4489795918367347,
+      "grad_norm": 0.7011855840682983,
+      "learning_rate": 0.0002,
+      "loss": 0.3052,
+      "step": 2820
+    },
+    {
+      "epoch": 2.457663916630482,
+      "grad_norm": 0.7993234992027283,
+      "learning_rate": 0.0002,
+      "loss": 0.3101,
+      "step": 2830
+    },
+    {
+      "epoch": 2.466348241424229,
+      "grad_norm": 0.7405799627304077,
+      "learning_rate": 0.0002,
+      "loss": 0.3067,
+      "step": 2840
+    },
+    {
+      "epoch": 2.4750325662179766,
+      "grad_norm": 0.6736032962799072,
+      "learning_rate": 0.0002,
+      "loss": 0.3017,
+      "step": 2850
+    },
+    {
+      "epoch": 2.4837168910117238,
+      "grad_norm": 0.6662697196006775,
+      "learning_rate": 0.0002,
+      "loss": 0.2991,
+      "step": 2860
+    },
+    {
+      "epoch": 2.4924012158054714,
+      "grad_norm": 0.7334403991699219,
+      "learning_rate": 0.0002,
+      "loss": 0.3047,
+      "step": 2870
+    },
+    {
+      "epoch": 2.5010855405992185,
+      "grad_norm": 1.1669126749038696,
+      "learning_rate": 0.0002,
+      "loss": 0.2906,
+      "step": 2880
+    },
+    {
+      "epoch": 2.5097698653929656,
+      "grad_norm": 0.8848074674606323,
+      "learning_rate": 0.0002,
+      "loss": 0.2967,
+      "step": 2890
+    },
+    {
+      "epoch": 2.518454190186713,
+      "grad_norm": 0.8354884386062622,
+      "learning_rate": 0.0002,
+      "loss": 0.3009,
+      "step": 2900
+    },
+    {
+      "epoch": 2.5271385149804604,
+      "grad_norm": 0.734761655330658,
+      "learning_rate": 0.0002,
+      "loss": 0.2907,
+      "step": 2910
+    },
+    {
+      "epoch": 2.5358228397742075,
+      "grad_norm": 0.73737633228302,
+      "learning_rate": 0.0002,
+      "loss": 0.2902,
+      "step": 2920
+    },
+    {
+      "epoch": 2.544507164567955,
+      "grad_norm": 1.2551928758621216,
+      "learning_rate": 0.0002,
+      "loss": 0.3023,
+      "step": 2930
+    },
+    {
+      "epoch": 2.5531914893617023,
+      "grad_norm": 0.8526908755302429,
+      "learning_rate": 0.0002,
+      "loss": 0.2978,
+      "step": 2940
+    },
+    {
+      "epoch": 2.5618758141554494,
+      "grad_norm": 0.7598440051078796,
+      "learning_rate": 0.0002,
+      "loss": 0.2998,
+      "step": 2950
+    },
+    {
+      "epoch": 2.5705601389491965,
+      "grad_norm": 0.4928401708602905,
+      "learning_rate": 0.0002,
+      "loss": 0.2927,
+      "step": 2960
+    },
+    {
+      "epoch": 2.579244463742944,
+      "grad_norm": 0.8329664468765259,
+      "learning_rate": 0.0002,
+      "loss": 0.2922,
+      "step": 2970
+    },
+    {
+      "epoch": 2.5879287885366913,
+      "grad_norm": 0.7364818453788757,
+      "learning_rate": 0.0002,
+      "loss": 0.2982,
+      "step": 2980
+    },
+    {
+      "epoch": 2.5966131133304384,
+      "grad_norm": 0.742920994758606,
+      "learning_rate": 0.0002,
+      "loss": 0.299,
+      "step": 2990
+    },
+    {
+      "epoch": 2.605297438124186,
+      "grad_norm": 0.6937646865844727,
+      "learning_rate": 0.0002,
+      "loss": 0.2963,
+      "step": 3000
+    },
+    {
+      "epoch": 2.613981762917933,
+      "grad_norm": 0.7259171009063721,
+      "learning_rate": 0.0002,
+      "loss": 0.2918,
+      "step": 3010
+    },
+    {
+      "epoch": 2.6226660877116803,
+      "grad_norm": 0.6129646301269531,
+      "learning_rate": 0.0002,
+      "loss": 0.2866,
+      "step": 3020
+    },
+    {
+      "epoch": 2.6313504125054274,
+      "grad_norm": 0.6982302069664001,
+      "learning_rate": 0.0002,
+      "loss": 0.29,
+      "step": 3030
+    },
+    {
+      "epoch": 2.640034737299175,
+      "grad_norm": 1.272072434425354,
+      "learning_rate": 0.0002,
+      "loss": 0.2968,
+      "step": 3040
+    },
+    {
+      "epoch": 2.648719062092922,
+      "grad_norm": 1.1105331182479858,
+      "learning_rate": 0.0002,
+      "loss": 0.3071,
+      "step": 3050
+    },
+    {
+      "epoch": 2.6574033868866698,
+      "grad_norm": 0.7434864044189453,
+      "learning_rate": 0.0002,
+      "loss": 0.2865,
+      "step": 3060
+    },
+    {
+      "epoch": 2.666087711680417,
+      "grad_norm": 0.8283593654632568,
+      "learning_rate": 0.0002,
+      "loss": 0.2973,
+      "step": 3070
+    },
+    {
+      "epoch": 2.674772036474164,
+      "grad_norm": 0.6259397268295288,
+      "learning_rate": 0.0002,
+      "loss": 0.292,
+      "step": 3080
+    },
+    {
+      "epoch": 2.683456361267911,
+      "grad_norm": 0.8509077429771423,
+      "learning_rate": 0.0002,
+      "loss": 0.29,
+      "step": 3090
+    },
+    {
+      "epoch": 2.692140686061659,
+      "grad_norm": 0.8883143067359924,
+      "learning_rate": 0.0002,
+      "loss": 0.2926,
+      "step": 3100
+    },
+    {
+      "epoch": 2.700825010855406,
+      "grad_norm": 0.7960503697395325,
+      "learning_rate": 0.0002,
+      "loss": 0.2928,
+      "step": 3110
+    },
+    {
+      "epoch": 2.7095093356491535,
+      "grad_norm": 0.7752923965454102,
+      "learning_rate": 0.0002,
+      "loss": 0.2935,
+      "step": 3120
+    },
+    {
+      "epoch": 2.7181936604429007,
+      "grad_norm": 0.6954398155212402,
+      "learning_rate": 0.0002,
+      "loss": 0.2819,
+      "step": 3130
+    },
+    {
+      "epoch": 2.726877985236648,
+      "grad_norm": 0.9302791953086853,
+      "learning_rate": 0.0002,
+      "loss": 0.2901,
+      "step": 3140
+    },
+    {
+      "epoch": 2.735562310030395,
+      "grad_norm": 0.7394571304321289,
+      "learning_rate": 0.0002,
+      "loss": 0.2941,
+      "step": 3150
+    },
+    {
+      "epoch": 2.7442466348241426,
+      "grad_norm": 0.7749691009521484,
+      "learning_rate": 0.0002,
+      "loss": 0.2865,
+      "step": 3160
+    },
+    {
+      "epoch": 2.7529309596178897,
+      "grad_norm": 0.8289293050765991,
+      "learning_rate": 0.0002,
+      "loss": 0.2906,
+      "step": 3170
+    },
+    {
+      "epoch": 2.761615284411637,
+      "grad_norm": 0.7008669376373291,
+      "learning_rate": 0.0002,
+      "loss": 0.2923,
+      "step": 3180
+    },
+    {
+      "epoch": 2.7702996092053844,
+      "grad_norm": 1.2574214935302734,
+      "learning_rate": 0.0002,
+      "loss": 0.2919,
+      "step": 3190
+    },
+    {
+      "epoch": 2.7789839339991316,
+      "grad_norm": 0.6361944675445557,
+      "learning_rate": 0.0002,
+      "loss": 0.29,
+      "step": 3200
+    },
+    {
+      "epoch": 2.7876682587928787,
+      "grad_norm": 0.6270219087600708,
+      "learning_rate": 0.0002,
+      "loss": 0.2848,
+      "step": 3210
+    },
+    {
+      "epoch": 2.7963525835866263,
+      "grad_norm": 0.5672947764396667,
+      "learning_rate": 0.0002,
+      "loss": 0.2895,
+      "step": 3220
+    },
+    {
+      "epoch": 2.8050369083803735,
+      "grad_norm": 0.8151955008506775,
+      "learning_rate": 0.0002,
+      "loss": 0.287,
+      "step": 3230
+    },
+    {
+      "epoch": 2.8137212331741206,
+      "grad_norm": 0.7757666110992432,
+      "learning_rate": 0.0002,
+      "loss": 0.295,
+      "step": 3240
+    },
+    {
+      "epoch": 2.822405557967868,
+      "grad_norm": 0.9094030857086182,
+      "learning_rate": 0.0002,
+      "loss": 0.2971,
+      "step": 3250
+    },
+    {
+      "epoch": 2.8310898827616153,
+      "grad_norm": 0.6486825346946716,
+      "learning_rate": 0.0002,
+      "loss": 0.2935,
+      "step": 3260
+    },
+    {
+      "epoch": 2.8397742075553625,
+      "grad_norm": 0.8660843968391418,
+      "learning_rate": 0.0002,
+      "loss": 0.288,
+      "step": 3270
+    },
+    {
+      "epoch": 2.8484585323491096,
+      "grad_norm": 0.6846482753753662,
+      "learning_rate": 0.0002,
+      "loss": 0.2785,
+      "step": 3280
+    },
+    {
+      "epoch": 2.857142857142857,
+      "grad_norm": 0.8217440247535706,
+      "learning_rate": 0.0002,
+      "loss": 0.2912,
+      "step": 3290
+    },
+    {
+      "epoch": 2.8658271819366044,
+      "grad_norm": 0.9265017509460449,
+      "learning_rate": 0.0002,
+      "loss": 0.2945,
+      "step": 3300
+    },
+    {
+      "epoch": 2.874511506730352,
+      "grad_norm": 0.6155156493186951,
+      "learning_rate": 0.0002,
+      "loss": 0.2967,
+      "step": 3310
+    },
+    {
+      "epoch": 2.883195831524099,
+      "grad_norm": 0.711634635925293,
+      "learning_rate": 0.0002,
+      "loss": 0.2922,
+      "step": 3320
+    },
+    {
+      "epoch": 2.8918801563178462,
+      "grad_norm": 0.7138526439666748,
+      "learning_rate": 0.0002,
+      "loss": 0.2829,
+      "step": 3330
+    },
+    {
+      "epoch": 2.9005644811115934,
+      "grad_norm": 0.8107818365097046,
+      "learning_rate": 0.0002,
+      "loss": 0.29,
+      "step": 3340
+    },
+    {
+      "epoch": 2.909248805905341,
+      "grad_norm": 1.169446587562561,
+      "learning_rate": 0.0002,
+      "loss": 0.2968,
+      "step": 3350
+    },
+    {
+      "epoch": 2.917933130699088,
+      "grad_norm": 0.8311055898666382,
+      "learning_rate": 0.0002,
+      "loss": 0.2844,
+      "step": 3360
+    },
+    {
+      "epoch": 2.9266174554928357,
+      "grad_norm": 0.741226077079773,
+      "learning_rate": 0.0002,
+      "loss": 0.2861,
+      "step": 3370
+    },
+    {
+      "epoch": 2.935301780286583,
+      "grad_norm": 0.6655572652816772,
+      "learning_rate": 0.0002,
+      "loss": 0.2861,
+      "step": 3380
+    },
+    {
+      "epoch": 2.94398610508033,
+      "grad_norm": 0.4534182846546173,
+      "learning_rate": 0.0002,
+      "loss": 0.2813,
+      "step": 3390
+    },
+    {
+      "epoch": 2.952670429874077,
+      "grad_norm": 0.8443124294281006,
+      "learning_rate": 0.0002,
+      "loss": 0.2899,
+      "step": 3400
+    },
+    {
+      "epoch": 2.9613547546678247,
+      "grad_norm": 0.748831570148468,
+      "learning_rate": 0.0002,
+      "loss": 0.2811,
+      "step": 3410
+    },
+    {
+      "epoch": 2.970039079461572,
+      "grad_norm": 0.6330915689468384,
+      "learning_rate": 0.0002,
+      "loss": 0.2842,
+      "step": 3420
+    },
+    {
+      "epoch": 2.978723404255319,
+      "grad_norm": 0.6212947964668274,
+      "learning_rate": 0.0002,
+      "loss": 0.2962,
+      "step": 3430
+    },
+    {
+      "epoch": 2.9874077290490666,
+      "grad_norm": 0.9319456815719604,
+      "learning_rate": 0.0002,
+      "loss": 0.2733,
+      "step": 3440
+    },
+    {
+      "epoch": 2.9960920538428137,
+      "grad_norm": 0.8269906044006348,
+      "learning_rate": 0.0002,
+      "loss": 0.2846,
+      "step": 3450
+    },
+    {
+      "epoch": 2.9995657837603127,
+      "eval_loss": 0.2924337387084961,
+      "eval_runtime": 51.4614,
+      "eval_samples_per_second": 7.093,
+      "eval_steps_per_second": 0.894,
+      "step": 3454
+    },
+    {
+      "epoch": 3.004776378636561,
+      "grad_norm": 0.6661036610603333,
+      "learning_rate": 0.0002,
+      "loss": 0.2768,
+      "step": 3460
+    },
+    {
+      "epoch": 3.0134607034303085,
+      "grad_norm": 0.7474643588066101,
+      "learning_rate": 0.0002,
+      "loss": 0.281,
+      "step": 3470
+    },
+    {
+      "epoch": 3.0221450282240556,
+      "grad_norm": 0.6831788420677185,
+      "learning_rate": 0.0002,
+      "loss": 0.281,
+      "step": 3480
+    },
+    {
+      "epoch": 3.0308293530178028,
+      "grad_norm": 1.0030725002288818,
+      "learning_rate": 0.0002,
+      "loss": 0.2759,
+      "step": 3490
+    },
+    {
+      "epoch": 3.0395136778115504,
+      "grad_norm": 0.6812708377838135,
+      "learning_rate": 0.0002,
+      "loss": 0.2709,
+      "step": 3500
+    },
+    {
+      "epoch": 3.0481980026052975,
+      "grad_norm": 0.5478284358978271,
+      "learning_rate": 0.0002,
+      "loss": 0.2673,
+      "step": 3510
+    },
+    {
+      "epoch": 3.0568823273990446,
+      "grad_norm": 0.72661292552948,
+      "learning_rate": 0.0002,
+      "loss": 0.283,
+      "step": 3520
+    },
+    {
+      "epoch": 3.065566652192792,
+      "grad_norm": 0.5697672963142395,
+      "learning_rate": 0.0002,
+      "loss": 0.2862,
+      "step": 3530
+    },
+    {
+      "epoch": 3.0742509769865394,
+      "grad_norm": 1.248355746269226,
+      "learning_rate": 0.0002,
+      "loss": 0.2726,
+      "step": 3540
+    },
+    {
+      "epoch": 3.0829353017802865,
+      "grad_norm": 0.6486856341362,
+      "learning_rate": 0.0002,
+      "loss": 0.2755,
+      "step": 3550
+    },
+    {
+      "epoch": 3.0916196265740337,
+      "grad_norm": 0.7223301529884338,
+      "learning_rate": 0.0002,
+      "loss": 0.2792,
+      "step": 3560
+    },
+    {
+      "epoch": 3.1003039513677813,
+      "grad_norm": 0.7293640971183777,
+      "learning_rate": 0.0002,
+      "loss": 0.2749,
+      "step": 3570
+    },
+    {
+      "epoch": 3.1089882761615284,
+      "grad_norm": 0.6899237036705017,
+      "learning_rate": 0.0002,
+      "loss": 0.2795,
+      "step": 3580
+    },
+    {
+      "epoch": 3.1176726009552755,
+      "grad_norm": 1.061281442642212,
+      "learning_rate": 0.0002,
+      "loss": 0.2744,
+      "step": 3590
+    },
+    {
+      "epoch": 3.126356925749023,
+      "grad_norm": 0.8757736682891846,
+      "learning_rate": 0.0002,
+      "loss": 0.2689,
+      "step": 3600
+    },
+    {
+      "epoch": 3.1350412505427703,
+      "grad_norm": 0.7929820418357849,
+      "learning_rate": 0.0002,
+      "loss": 0.2793,
+      "step": 3610
+    },
+    {
+      "epoch": 3.1437255753365174,
+      "grad_norm": 0.793014407157898,
+      "learning_rate": 0.0002,
+      "loss": 0.2729,
+      "step": 3620
+    },
+    {
+      "epoch": 3.152409900130265,
+      "grad_norm": 1.0920517444610596,
+      "learning_rate": 0.0002,
+      "loss": 0.2742,
+      "step": 3630
+    },
+    {
+      "epoch": 3.161094224924012,
+      "grad_norm": 0.7001246809959412,
+      "learning_rate": 0.0002,
+      "loss": 0.277,
+      "step": 3640
+    },
+    {
+      "epoch": 3.1697785497177593,
+      "grad_norm": 0.6595224142074585,
+      "learning_rate": 0.0002,
+      "loss": 0.2724,
+      "step": 3650
+    },
+    {
+      "epoch": 3.178462874511507,
+      "grad_norm": 0.6215218305587769,
+      "learning_rate": 0.0002,
+      "loss": 0.2674,
+      "step": 3660
+    },
+    {
+      "epoch": 3.187147199305254,
+      "grad_norm": 0.6349411606788635,
+      "learning_rate": 0.0002,
+      "loss": 0.2834,
+      "step": 3670
+    },
+    {
+      "epoch": 3.195831524099001,
+      "grad_norm": 0.6490129232406616,
+      "learning_rate": 0.0002,
+      "loss": 0.2783,
+      "step": 3680
+    },
+    {
+      "epoch": 3.2045158488927488,
+      "grad_norm": 0.8096078038215637,
+      "learning_rate": 0.0002,
+      "loss": 0.2699,
+      "step": 3690
+    },
+    {
+      "epoch": 3.213200173686496,
+      "grad_norm": 0.5069501399993896,
+      "learning_rate": 0.0002,
+      "loss": 0.2731,
+      "step": 3700
+    },
+    {
+      "epoch": 3.221884498480243,
+      "grad_norm": 0.9933004379272461,
+      "learning_rate": 0.0002,
+      "loss": 0.2865,
+      "step": 3710
+    },
+    {
+      "epoch": 3.2305688232739906,
+      "grad_norm": 0.4949461817741394,
+      "learning_rate": 0.0002,
+      "loss": 0.2816,
+      "step": 3720
+    },
+    {
+      "epoch": 3.239253148067738,
+      "grad_norm": 0.8595077991485596,
+      "learning_rate": 0.0002,
+      "loss": 0.2712,
+      "step": 3730
+    },
+    {
+      "epoch": 3.247937472861485,
+      "grad_norm": 0.9714534282684326,
+      "learning_rate": 0.0002,
+      "loss": 0.2746,
+      "step": 3740
+    },
+    {
+      "epoch": 3.2566217976552325,
+      "grad_norm": 0.5247510671615601,
+      "learning_rate": 0.0002,
+      "loss": 0.2817,
+      "step": 3750
+    },
+    {
+      "epoch": 3.2653061224489797,
+      "grad_norm": 0.6707687973976135,
+      "learning_rate": 0.0002,
+      "loss": 0.2727,
+      "step": 3760
+    },
+    {
+      "epoch": 3.273990447242727,
+      "grad_norm": 0.6039386987686157,
+      "learning_rate": 0.0002,
+      "loss": 0.2823,
+      "step": 3770
+    },
+    {
+      "epoch": 3.282674772036474,
+      "grad_norm": 0.7876906991004944,
+      "learning_rate": 0.0002,
+      "loss": 0.2837,
+      "step": 3780
+    },
+    {
+      "epoch": 3.2913590968302215,
+      "grad_norm": 0.7849938869476318,
+      "learning_rate": 0.0002,
+      "loss": 0.2778,
+      "step": 3790
+    },
+    {
+      "epoch": 3.3000434216239687,
+      "grad_norm": 0.9591494798660278,
+      "learning_rate": 0.0002,
+      "loss": 0.2767,
+      "step": 3800
+    },
+    {
+      "epoch": 3.308727746417716,
+      "grad_norm": 0.5763871669769287,
+      "learning_rate": 0.0002,
+      "loss": 0.2679,
+      "step": 3810
+    },
+    {
+      "epoch": 3.3174120712114634,
+      "grad_norm": 0.635368824005127,
+      "learning_rate": 0.0002,
+      "loss": 0.2662,
+      "step": 3820
+    },
+    {
+      "epoch": 3.3260963960052106,
+      "grad_norm": 0.9177204370498657,
+      "learning_rate": 0.0002,
+      "loss": 0.2773,
+      "step": 3830
+    },
+    {
+      "epoch": 3.3347807207989577,
+      "grad_norm": 0.8839341402053833,
+      "learning_rate": 0.0002,
+      "loss": 0.2789,
+      "step": 3840
+    },
+    {
+      "epoch": 3.3434650455927053,
+      "grad_norm": 0.7208172678947449,
+      "learning_rate": 0.0002,
+      "loss": 0.2621,
+      "step": 3850
+    },
+    {
+      "epoch": 3.3521493703864524,
+      "grad_norm": 0.7215853333473206,
+      "learning_rate": 0.0002,
+      "loss": 0.2722,
+      "step": 3860
+    },
+    {
+      "epoch": 3.3608336951801996,
+      "grad_norm": 0.6294577121734619,
+      "learning_rate": 0.0002,
+      "loss": 0.2672,
+      "step": 3870
+    },
+    {
+      "epoch": 3.369518019973947,
+      "grad_norm": 0.5451186299324036,
+      "learning_rate": 0.0002,
+      "loss": 0.2707,
+      "step": 3880
+    },
+    {
+      "epoch": 3.3782023447676943,
+      "grad_norm": 0.8365423083305359,
+      "learning_rate": 0.0002,
+      "loss": 0.2762,
+      "step": 3890
+    },
+    {
+      "epoch": 3.3868866695614415,
+      "grad_norm": 0.39549630880355835,
+      "learning_rate": 0.0002,
+      "loss": 0.2543,
+      "step": 3900
+    },
+    {
+      "epoch": 3.395570994355189,
+      "grad_norm": 0.5622147917747498,
+      "learning_rate": 0.0002,
+      "loss": 0.27,
+      "step": 3910
+    },
+    {
+      "epoch": 3.404255319148936,
+      "grad_norm": 0.8139942288398743,
+      "learning_rate": 0.0002,
+      "loss": 0.2842,
+      "step": 3920
+    },
+    {
+      "epoch": 3.4129396439426833,
+      "grad_norm": 0.9032343626022339,
+      "learning_rate": 0.0002,
+      "loss": 0.2811,
+      "step": 3930
+    },
+    {
+      "epoch": 3.421623968736431,
+      "grad_norm": 1.3641077280044556,
+      "learning_rate": 0.0002,
+      "loss": 0.2781,
+      "step": 3940
+    },
+    {
+      "epoch": 3.430308293530178,
+      "grad_norm": 0.5022559762001038,
+      "learning_rate": 0.0002,
+      "loss": 0.2775,
+      "step": 3950
+    },
+    {
+      "epoch": 3.438992618323925,
+      "grad_norm": 0.8359620571136475,
+      "learning_rate": 0.0002,
+      "loss": 0.2805,
+      "step": 3960
+    },
+    {
+      "epoch": 3.4476769431176724,
+      "grad_norm": 0.9538311958312988,
+      "learning_rate": 0.0002,
+      "loss": 0.2622,
+      "step": 3970
+    },
+    {
+      "epoch": 3.45636126791142,
+      "grad_norm": 0.7387760877609253,
+      "learning_rate": 0.0002,
+      "loss": 0.2759,
+      "step": 3980
+    },
+    {
+      "epoch": 3.465045592705167,
+      "grad_norm": 0.5856384634971619,
+      "learning_rate": 0.0002,
+      "loss": 0.274,
+      "step": 3990
+    },
+    {
+      "epoch": 3.4737299174989147,
+      "grad_norm": 0.7706475853919983,
+      "learning_rate": 0.0002,
+      "loss": 0.2736,
+      "step": 4000
+    },
+    {
+      "epoch": 3.482414242292662,
+      "grad_norm": 0.9640643000602722,
+      "learning_rate": 0.0002,
+      "loss": 0.27,
+      "step": 4010
+    },
+    {
+      "epoch": 3.491098567086409,
+      "grad_norm": 0.6987433433532715,
+      "learning_rate": 0.0002,
+      "loss": 0.2666,
+      "step": 4020
+    },
+    {
+      "epoch": 3.499782891880156,
+      "grad_norm": 0.7158568501472473,
+      "learning_rate": 0.0002,
+      "loss": 0.2764,
+      "step": 4030
+    },
+    {
+      "epoch": 3.5084672166739037,
+      "grad_norm": 0.7418273091316223,
+      "learning_rate": 0.0002,
+      "loss": 0.2706,
+      "step": 4040
+    },
+    {
+      "epoch": 3.517151541467651,
+      "grad_norm": 0.7849529981613159,
+      "learning_rate": 0.0002,
+      "loss": 0.2709,
+      "step": 4050
+    },
+    {
+      "epoch": 3.5258358662613984,
+      "grad_norm": 0.7308112978935242,
+      "learning_rate": 0.0002,
+      "loss": 0.2704,
+      "step": 4060
+    },
+    {
+      "epoch": 3.5345201910551456,
+      "grad_norm": 0.8360651135444641,
+      "learning_rate": 0.0002,
+      "loss": 0.2727,
+      "step": 4070
+    },
+    {
+      "epoch": 3.5432045158488927,
+      "grad_norm": 0.676218569278717,
+      "learning_rate": 0.0002,
+      "loss": 0.272,
+      "step": 4080
+    },
+    {
+      "epoch": 3.55188884064264,
+      "grad_norm": 0.5785254240036011,
+      "learning_rate": 0.0002,
+      "loss": 0.2704,
+      "step": 4090
+    },
+    {
+      "epoch": 3.5605731654363875,
+      "grad_norm": 1.0767836570739746,
+      "learning_rate": 0.0002,
+      "loss": 0.2669,
+      "step": 4100
+    },
+    {
+      "epoch": 3.5692574902301346,
+      "grad_norm": 0.7767327427864075,
+      "learning_rate": 0.0002,
+      "loss": 0.2666,
+      "step": 4110
+    },
+    {
+      "epoch": 3.5779418150238818,
+      "grad_norm": 0.7035610675811768,
+      "learning_rate": 0.0002,
+      "loss": 0.2752,
+      "step": 4120
+    },
+    {
+      "epoch": 3.5866261398176293,
+      "grad_norm": 0.6232367753982544,
+      "learning_rate": 0.0002,
+      "loss": 0.2759,
+      "step": 4130
+    },
+    {
+      "epoch": 3.5953104646113765,
+      "grad_norm": 0.8853907585144043,
+      "learning_rate": 0.0002,
+      "loss": 0.2635,
+      "step": 4140
+    },
+    {
+      "epoch": 3.6039947894051236,
+      "grad_norm": 0.6743808388710022,
+      "learning_rate": 0.0002,
+      "loss": 0.2712,
+      "step": 4150
+    },
+    {
+      "epoch": 3.6126791141988708,
+      "grad_norm": 0.5065876841545105,
+      "learning_rate": 0.0002,
+      "loss": 0.269,
+      "step": 4160
+    },
+    {
+      "epoch": 3.6213634389926184,
+      "grad_norm": 1.208004117012024,
+      "learning_rate": 0.0002,
+      "loss": 0.2707,
+      "step": 4170
+    },
+    {
+      "epoch": 3.6300477637863655,
+      "grad_norm": 0.7576237916946411,
+      "learning_rate": 0.0002,
+      "loss": 0.2647,
+      "step": 4180
+    },
+    {
+      "epoch": 3.638732088580113,
+      "grad_norm": 0.7501868605613708,
+      "learning_rate": 0.0002,
+      "loss": 0.2697,
+      "step": 4190
+    },
+    {
+      "epoch": 3.6474164133738602,
+      "grad_norm": 0.8302589058876038,
+      "learning_rate": 0.0002,
+      "loss": 0.2724,
+      "step": 4200
+    },
+    {
+      "epoch": 3.6561007381676074,
+      "grad_norm": 0.7079733610153198,
+      "learning_rate": 0.0002,
+      "loss": 0.2636,
+      "step": 4210
+    },
+    {
+      "epoch": 3.6647850629613545,
+      "grad_norm": 0.808874785900116,
+      "learning_rate": 0.0002,
+      "loss": 0.2666,
+      "step": 4220
+    },
+    {
+      "epoch": 3.673469387755102,
+      "grad_norm": 0.8256613612174988,
+      "learning_rate": 0.0002,
+      "loss": 0.272,
+      "step": 4230
+    },
+    {
+      "epoch": 3.6821537125488493,
+      "grad_norm": 0.5885641574859619,
+      "learning_rate": 0.0002,
+      "loss": 0.2677,
+      "step": 4240
+    },
+    {
+      "epoch": 3.690838037342597,
+      "grad_norm": 0.5877922177314758,
+      "learning_rate": 0.0002,
+      "loss": 0.265,
+      "step": 4250
+    },
+    {
+      "epoch": 3.699522362136344,
+      "grad_norm": 0.6538875699043274,
+      "learning_rate": 0.0002,
+      "loss": 0.2644,
+      "step": 4260
+    },
+    {
+      "epoch": 3.708206686930091,
+      "grad_norm": 0.5613241195678711,
+      "learning_rate": 0.0002,
+      "loss": 0.2711,
+      "step": 4270
+    },
+    {
+      "epoch": 3.7168910117238383,
+      "grad_norm": 1.0037645101547241,
+      "learning_rate": 0.0002,
+      "loss": 0.277,
+      "step": 4280
+    },
+    {
+      "epoch": 3.725575336517586,
+      "grad_norm": 0.7208364009857178,
+      "learning_rate": 0.0002,
+      "loss": 0.2676,
+      "step": 4290
+    },
+    {
+      "epoch": 3.734259661311333,
+      "grad_norm": 0.751794159412384,
+      "learning_rate": 0.0002,
+      "loss": 0.265,
+      "step": 4300
+    },
+    {
+      "epoch": 3.74294398610508,
+      "grad_norm": 0.45062699913978577,
+      "learning_rate": 0.0002,
+      "loss": 0.2672,
+      "step": 4310
+    },
+    {
+      "epoch": 3.7516283108988278,
+      "grad_norm": 0.657162606716156,
+      "learning_rate": 0.0002,
+      "loss": 0.2722,
+      "step": 4320
+    },
+    {
+      "epoch": 3.760312635692575,
+      "grad_norm": 0.6693727970123291,
+      "learning_rate": 0.0002,
+      "loss": 0.2726,
+      "step": 4330
+    },
+    {
+      "epoch": 3.768996960486322,
+      "grad_norm": 0.5274078249931335,
+      "learning_rate": 0.0002,
+      "loss": 0.2694,
+      "step": 4340
+    },
+    {
+      "epoch": 3.7776812852800696,
+      "grad_norm": 0.7587695717811584,
+      "learning_rate": 0.0002,
+      "loss": 0.2663,
+      "step": 4350
+    },
+    {
+      "epoch": 3.786365610073817,
+      "grad_norm": 0.49403151869773865,
+      "learning_rate": 0.0002,
+      "loss": 0.2755,
+      "step": 4360
+    },
+    {
+      "epoch": 3.795049934867564,
+      "grad_norm": 1.2706613540649414,
+      "learning_rate": 0.0002,
+      "loss": 0.2699,
+      "step": 4370
+    },
+    {
+      "epoch": 3.8037342596613115,
+      "grad_norm": 0.9016162157058716,
+      "learning_rate": 0.0002,
+      "loss": 0.2707,
+      "step": 4380
+    },
+    {
+      "epoch": 3.8124185844550587,
+      "grad_norm": 0.8027393221855164,
+      "learning_rate": 0.0002,
+      "loss": 0.273,
+      "step": 4390
+    },
+    {
+      "epoch": 3.821102909248806,
+      "grad_norm": 0.49945929646492004,
+      "learning_rate": 0.0002,
+      "loss": 0.2732,
+      "step": 4400
+    },
+    {
+      "epoch": 3.829787234042553,
+      "grad_norm": 0.775516927242279,
+      "learning_rate": 0.0002,
+      "loss": 0.2716,
+      "step": 4410
+    },
+    {
+      "epoch": 3.8384715588363005,
+      "grad_norm": 0.5034677982330322,
+      "learning_rate": 0.0002,
+      "loss": 0.2665,
+      "step": 4420
+    },
+    {
+      "epoch": 3.8471558836300477,
+      "grad_norm": 0.670738935470581,
+      "learning_rate": 0.0002,
+      "loss": 0.2678,
+      "step": 4430
+    },
+    {
+      "epoch": 3.8558402084237953,
+      "grad_norm": 0.6182120442390442,
+      "learning_rate": 0.0002,
+      "loss": 0.2709,
+      "step": 4440
+    },
+    {
+      "epoch": 3.8645245332175424,
+      "grad_norm": 0.761558473110199,
+      "learning_rate": 0.0002,
+      "loss": 0.2727,
+      "step": 4450
+    },
+    {
+      "epoch": 3.8732088580112896,
+      "grad_norm": 0.7241739630699158,
+      "learning_rate": 0.0002,
+      "loss": 0.2643,
+      "step": 4460
+    },
+    {
+      "epoch": 3.8818931828050367,
+      "grad_norm": 0.6606132388114929,
+      "learning_rate": 0.0002,
+      "loss": 0.2644,
+      "step": 4470
+    },
+    {
+      "epoch": 3.8905775075987843,
+      "grad_norm": 0.6781331896781921,
+      "learning_rate": 0.0002,
+      "loss": 0.2617,
+      "step": 4480
+    },
+    {
+      "epoch": 3.8992618323925314,
+      "grad_norm": 0.6857254505157471,
+      "learning_rate": 0.0002,
+      "loss": 0.264,
+      "step": 4490
+    },
+    {
+      "epoch": 3.907946157186279,
+      "grad_norm": 0.6594447493553162,
+      "learning_rate": 0.0002,
+      "loss": 0.2737,
+      "step": 4500
+    },
+    {
+      "epoch": 3.916630481980026,
+      "grad_norm": 1.2038878202438354,
+      "learning_rate": 0.0002,
+      "loss": 0.2685,
+      "step": 4510
+    },
+    {
+      "epoch": 3.9253148067737733,
+      "grad_norm": 1.227657675743103,
+      "learning_rate": 0.0002,
+      "loss": 0.2761,
+      "step": 4520
+    },
+    {
+      "epoch": 3.9339991315675205,
+      "grad_norm": 0.7535707950592041,
+      "learning_rate": 0.0002,
+      "loss": 0.2702,
+      "step": 4530
+    },
+    {
+      "epoch": 3.942683456361268,
+      "grad_norm": 0.8086929321289062,
+      "learning_rate": 0.0002,
+      "loss": 0.2679,
+      "step": 4540
+    },
+    {
+      "epoch": 3.951367781155015,
+      "grad_norm": 0.5432729721069336,
+      "learning_rate": 0.0002,
+      "loss": 0.2616,
+      "step": 4550
+    },
+    {
+      "epoch": 3.9600521059487623,
+      "grad_norm": 0.6326302886009216,
+      "learning_rate": 0.0002,
+      "loss": 0.2606,
+      "step": 4560
+    },
+    {
+      "epoch": 3.96873643074251,
+      "grad_norm": 0.7258507013320923,
+      "learning_rate": 0.0002,
+      "loss": 0.269,
+      "step": 4570
+    },
+    {
+      "epoch": 3.977420755536257,
+      "grad_norm": 0.5267454385757446,
+      "learning_rate": 0.0002,
+      "loss": 0.2732,
+      "step": 4580
+    },
+    {
+      "epoch": 3.986105080330004,
+      "grad_norm": 0.6834063529968262,
+      "learning_rate": 0.0002,
+      "loss": 0.2649,
+      "step": 4590
+    },
+    {
+      "epoch": 3.9947894051237514,
+      "grad_norm": 0.4831315279006958,
+      "learning_rate": 0.0002,
+      "loss": 0.2644,
+      "step": 4600
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.2722988724708557,
+      "eval_runtime": 51.4695,
+      "eval_samples_per_second": 7.092,
+      "eval_steps_per_second": 0.894,
+      "step": 4606
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 9208,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.020869912133632e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}