Add files using upload-large-folder tool

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: transformers
+model_name: Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
+---
+
+# Model Card for Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test1
+
+This model is a fine-tuned version of [Qwen/Qwen3-14B-Base](https://huggingface.co/Qwen/Qwen3-14B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/u51ovefm) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: transformers
+model_name: Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
+---
+
+# Model Card for Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2
+
+This model is a fine-tuned version of [Qwen/Qwen3-14B-Base](https://huggingface.co/Qwen/Qwen3-14B-Base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/katriin-kukk/Cross_lingual_morphological_generalization/runs/q6nyoshd) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 0.29.0
+- Transformers: 5.5.4
+- Pytorch: 2.10.0
+- Datasets: 4.6.1
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-100/trainer_state.json

@@ -0,0 +1,139 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.24125452352231605,
+  "eval_steps": 20,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.0561259958272e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1000/trainer_state.json

@@ -0,0 +1,1084 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.4101326899879374,
+  "eval_steps": 20,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    },
+    {
+      "entropy": 0.8518766984343529,
+      "epoch": 0.28950542822677927,
+      "grad_norm": 0.635079026222229,
+      "learning_rate": 2.0729210702058663e-05,
+      "loss": 0.8058656692504883,
+      "mean_token_accuracy": 0.7847469851374627,
+      "num_tokens": 298041.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28950542822677927,
+      "eval_entropy": 0.838603059227547,
+      "eval_loss": 0.7992516756057739,
+      "eval_mean_token_accuracy": 0.7849764954508021,
+      "eval_num_tokens": 298041.0,
+      "eval_runtime": 98.5627,
+      "eval_samples_per_second": 14.407,
+      "eval_steps_per_second": 1.806,
+      "step": 120
+    },
+    {
+      "entropy": 0.8069365546107292,
+      "epoch": 0.33775633293124246,
+      "grad_norm": 0.5280472636222839,
+      "learning_rate": 2.421311166038785e-05,
+      "loss": 0.7562496662139893,
+      "mean_token_accuracy": 0.7897997036576271,
+      "num_tokens": 348955.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33775633293124246,
+      "eval_entropy": 0.8161682168419442,
+      "eval_loss": 0.7599642276763916,
+      "eval_mean_token_accuracy": 0.7912511644738444,
+      "eval_num_tokens": 348955.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 140
+    },
+    {
+      "entropy": 0.7952415809035301,
+      "epoch": 0.38600723763570566,
+      "grad_norm": 0.5387083888053894,
+      "learning_rate": 2.7697012618717034e-05,
+      "loss": 0.740808629989624,
+      "mean_token_accuracy": 0.7941806614398956,
+      "num_tokens": 395798.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38600723763570566,
+      "eval_entropy": 0.7856516191798649,
+      "eval_loss": 0.7324908971786499,
+      "eval_mean_token_accuracy": 0.7980317825011993,
+      "eval_num_tokens": 395798.0,
+      "eval_runtime": 98.6378,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 160
+    },
+    {
+      "entropy": 0.7537026956677437,
+      "epoch": 0.43425814234016885,
+      "grad_norm": 0.5521425604820251,
+      "learning_rate": 3.118091357704622e-05,
+      "loss": 0.7073616027832031,
+      "mean_token_accuracy": 0.8006261065602303,
+      "num_tokens": 446437.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43425814234016885,
+      "eval_entropy": 0.7470693678668375,
+      "eval_loss": 0.711143434047699,
+      "eval_mean_token_accuracy": 0.801459076029531,
+      "eval_num_tokens": 446437.0,
+      "eval_runtime": 98.6667,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 180
+    },
+    {
+      "entropy": 0.7328549668192863,
+      "epoch": 0.4825090470446321,
+      "grad_norm": 0.4901895821094513,
+      "learning_rate": 3.4664814535375406e-05,
+      "loss": 0.6874266624450683,
+      "mean_token_accuracy": 0.8066758885979652,
+      "num_tokens": 497682.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4825090470446321,
+      "eval_entropy": 0.7374835154983435,
+      "eval_loss": 0.6967977285385132,
+      "eval_mean_token_accuracy": 0.8055616520094068,
+      "eval_num_tokens": 497682.0,
+      "eval_runtime": 98.6234,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 200
+    },
+    {
+      "entropy": 0.7372006759047508,
+      "epoch": 0.5307599517490953,
+      "grad_norm": 0.5516473650932312,
+      "learning_rate": 3.814871549370459e-05,
+      "loss": 0.6906380653381348,
+      "mean_token_accuracy": 0.8057604253292083,
+      "num_tokens": 544670.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5307599517490953,
+      "eval_entropy": 0.710068748573239,
+      "eval_loss": 0.6860950589179993,
+      "eval_mean_token_accuracy": 0.8069355440943429,
+      "eval_num_tokens": 544670.0,
+      "eval_runtime": 98.4618,
+      "eval_samples_per_second": 14.422,
+      "eval_steps_per_second": 1.808,
+      "step": 220
+    },
+    {
+      "entropy": 0.7099859565496445,
+      "epoch": 0.5790108564535585,
+      "grad_norm": 0.457378625869751,
+      "learning_rate": 4.1632616452033785e-05,
+      "loss": 0.6674468517303467,
+      "mean_token_accuracy": 0.8087703332304954,
+      "num_tokens": 592926.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5790108564535585,
+      "eval_entropy": 0.717441116156203,
+      "eval_loss": 0.6765118837356567,
+      "eval_mean_token_accuracy": 0.8095786246690857,
+      "eval_num_tokens": 592926.0,
+      "eval_runtime": 98.6828,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 240
+    },
+    {
+      "entropy": 0.7170855522155761,
+      "epoch": 0.6272617611580217,
+      "grad_norm": 0.4604854881763458,
+      "learning_rate": 4.5116517410362974e-05,
+      "loss": 0.6647133827209473,
+      "mean_token_accuracy": 0.8115006521344185,
+      "num_tokens": 639077.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6272617611580217,
+      "eval_entropy": 0.7082702219486237,
+      "eval_loss": 0.6722468733787537,
+      "eval_mean_token_accuracy": 0.8100237367528208,
+      "eval_num_tokens": 639077.0,
+      "eval_runtime": 98.6242,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 260
+    },
+    {
+      "entropy": 0.691286925971508,
+      "epoch": 0.6755126658624849,
+      "grad_norm": 0.4626481831073761,
+      "learning_rate": 4.860041836869215e-05,
+      "loss": 0.6543298721313476,
+      "mean_token_accuracy": 0.8142751976847649,
+      "num_tokens": 691028.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6755126658624849,
+      "eval_entropy": 0.7341568747932992,
+      "eval_loss": 0.658997654914856,
+      "eval_mean_token_accuracy": 0.813439255684949,
+      "eval_num_tokens": 691028.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 280
+    },
+    {
+      "entropy": 0.691513468325138,
+      "epoch": 0.7237635705669482,
+      "grad_norm": 0.5434145331382751,
+      "learning_rate": 5.2084319327021346e-05,
+      "loss": 0.6393396377563476,
+      "mean_token_accuracy": 0.8172704800963402,
+      "num_tokens": 740255.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7237635705669482,
+      "eval_entropy": 0.699160492487168,
+      "eval_loss": 0.6522852182388306,
+      "eval_mean_token_accuracy": 0.8143363323774231,
+      "eval_num_tokens": 740255.0,
+      "eval_runtime": 98.6233,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 300
+    },
+    {
+      "entropy": 0.6918429866433143,
+      "epoch": 0.7720144752714113,
+      "grad_norm": 0.47131890058517456,
+      "learning_rate": 5.556822028535052e-05,
+      "loss": 0.6503581523895263,
+      "mean_token_accuracy": 0.8149650946259499,
+      "num_tokens": 789529.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7720144752714113,
+      "eval_entropy": 0.6949931701247611,
+      "eval_loss": 0.6464414596557617,
+      "eval_mean_token_accuracy": 0.816310279489903,
+      "eval_num_tokens": 789529.0,
+      "eval_runtime": 98.4759,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 320
+    },
+    {
+      "entropy": 0.6926439955830574,
+      "epoch": 0.8202653799758746,
+      "grad_norm": 0.46790942549705505,
+      "learning_rate": 5.905212124367972e-05,
+      "loss": 0.6453651428222656,
+      "mean_token_accuracy": 0.8171535924077034,
+      "num_tokens": 841584.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8202653799758746,
+      "eval_entropy": 0.6810062698433909,
+      "eval_loss": 0.6440736651420593,
+      "eval_mean_token_accuracy": 0.8166696479481258,
+      "eval_num_tokens": 841584.0,
+      "eval_runtime": 98.6479,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 340
+    },
+    {
+      "entropy": 0.7038845136761666,
+      "epoch": 0.8685162846803377,
+      "grad_norm": 0.5038695335388184,
+      "learning_rate": 6.253602220200891e-05,
+      "loss": 0.6558614253997803,
+      "mean_token_accuracy": 0.8127013072371483,
+      "num_tokens": 885769.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8685162846803377,
+      "eval_entropy": 0.6752923710292644,
+      "eval_loss": 0.6410524845123291,
+      "eval_mean_token_accuracy": 0.817534717950928,
+      "eval_num_tokens": 885769.0,
+      "eval_runtime": 98.5986,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 360
+    },
+    {
+      "entropy": 0.6897840186953544,
+      "epoch": 0.916767189384801,
+      "grad_norm": 0.5168097615242004,
+      "learning_rate": 6.601992316033808e-05,
+      "loss": 0.6328047752380371,
+      "mean_token_accuracy": 0.8181755512952804,
+      "num_tokens": 930056.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.916767189384801,
+      "eval_entropy": 0.6816390936964014,
+      "eval_loss": 0.6402509212493896,
+      "eval_mean_token_accuracy": 0.8172605586855599,
+      "eval_num_tokens": 930056.0,
+      "eval_runtime": 98.4787,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 380
+    },
+    {
+      "entropy": 0.6798740655183793,
+      "epoch": 0.9650180940892642,
+      "grad_norm": 0.4609147906303406,
+      "learning_rate": 6.950382411866727e-05,
+      "loss": 0.6358067989349365,
+      "mean_token_accuracy": 0.8193414479494094,
+      "num_tokens": 976892.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9650180940892642,
+      "eval_entropy": 0.6841439577970612,
+      "eval_loss": 0.6335230469703674,
+      "eval_mean_token_accuracy": 0.8190797965848045,
+      "eval_num_tokens": 976892.0,
+      "eval_runtime": 98.5983,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 400
+    },
+    {
+      "entropy": 0.6647876103719076,
+      "epoch": 1.0120627261761157,
+      "grad_norm": 0.46518462896347046,
+      "learning_rate": 7.229074030587624e-05,
+      "loss": 0.6112009525299072,
+      "mean_token_accuracy": 0.8212849283829714,
+      "num_tokens": 1024472.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0120627261761157,
+      "eval_entropy": 0.6612423688508151,
+      "eval_loss": 0.6311497688293457,
+      "eval_mean_token_accuracy": 0.8194578638237514,
+      "eval_num_tokens": 1024472.0,
+      "eval_runtime": 98.4872,
+      "eval_samples_per_second": 14.418,
+      "eval_steps_per_second": 1.807,
+      "step": 420
+    },
+    {
+      "entropy": 0.6288895070552826,
+      "epoch": 1.060313630880579,
+      "grad_norm": 0.4342021644115448,
+      "learning_rate": 7.228358026812833e-05,
+      "loss": 0.5853999614715576,
+      "mean_token_accuracy": 0.82901301831007,
+      "num_tokens": 1076964.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060313630880579,
+      "eval_entropy": 0.6567474329069759,
+      "eval_loss": 0.6245927810668945,
+      "eval_mean_token_accuracy": 0.8211770050981072,
+      "eval_num_tokens": 1076964.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 440
+    },
+    {
+      "entropy": 0.639747454226017,
+      "epoch": 1.1085645355850422,
+      "grad_norm": 0.5338298082351685,
+      "learning_rate": 7.226619357333845e-05,
+      "loss": 0.5847910404205322,
+      "mean_token_accuracy": 0.8279331669211387,
+      "num_tokens": 1126064.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1085645355850422,
+      "eval_entropy": 0.6399250211340658,
+      "eval_loss": 0.6245712637901306,
+      "eval_mean_token_accuracy": 0.8209733875949731,
+      "eval_num_tokens": 1126064.0,
+      "eval_runtime": 98.6581,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 460
+    },
+    {
+      "entropy": 0.6248803421854973,
+      "epoch": 1.1568154402895054,
+      "grad_norm": 0.4402505159378052,
+      "learning_rate": 7.223858514173552e-05,
+      "loss": 0.5847790241241455,
+      "mean_token_accuracy": 0.8273933246731758,
+      "num_tokens": 1176912.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1568154402895054,
+      "eval_entropy": 0.6459465750147787,
+      "eval_loss": 0.615716278553009,
+      "eval_mean_token_accuracy": 0.8228987813665626,
+      "eval_num_tokens": 1176912.0,
+      "eval_runtime": 98.5987,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 480
+    },
+    {
+      "entropy": 0.6356995061039925,
+      "epoch": 1.2050663449939687,
+      "grad_norm": 0.5318694114685059,
+      "learning_rate": 7.220076278617914e-05,
+      "loss": 0.5902805328369141,
+      "mean_token_accuracy": 0.8279137581586837,
+      "num_tokens": 1223383.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2050663449939687,
+      "eval_entropy": 0.6509443488013879,
+      "eval_loss": 0.6174858808517456,
+      "eval_mean_token_accuracy": 0.8229377031996009,
+      "eval_num_tokens": 1223383.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 500
+    },
+    {
+      "entropy": 0.6378119193017483,
+      "epoch": 1.2533172496984317,
+      "grad_norm": 0.5385730266571045,
+      "learning_rate": 7.215273720994853e-05,
+      "loss": 0.5850183486938476,
+      "mean_token_accuracy": 0.8282771646976471,
+      "num_tokens": 1273534.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2533172496984317,
+      "eval_entropy": 0.6438969477508845,
+      "eval_loss": 0.6134508848190308,
+      "eval_mean_token_accuracy": 0.823984161186754,
+      "eval_num_tokens": 1273534.0,
+      "eval_runtime": 98.4814,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 520
+    },
+    {
+      "entropy": 0.6330174207687378,
+      "epoch": 1.301568154402895,
+      "grad_norm": 0.4950766861438751,
+      "learning_rate": 7.209452200371367e-05,
+      "loss": 0.5886627674102783,
+      "mean_token_accuracy": 0.828557687997818,
+      "num_tokens": 1323351.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.301568154402895,
+      "eval_entropy": 0.6342737025089478,
+      "eval_loss": 0.611145555973053,
+      "eval_mean_token_accuracy": 0.8250270244780551,
+      "eval_num_tokens": 1323351.0,
+      "eval_runtime": 98.6698,
+      "eval_samples_per_second": 14.391,
+      "eval_steps_per_second": 1.804,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270076125860214,
+      "epoch": 1.3498190591073582,
+      "grad_norm": 0.435604989528656,
+      "learning_rate": 7.20261336416893e-05,
+      "loss": 0.5834732532501221,
+      "mean_token_accuracy": 0.8290600389242172,
+      "num_tokens": 1371508.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3498190591073582,
+      "eval_entropy": 0.6624904408883513,
+      "eval_loss": 0.6063718795776367,
+      "eval_mean_token_accuracy": 0.825260305002834,
+      "eval_num_tokens": 1371508.0,
+      "eval_runtime": 98.6348,
+      "eval_samples_per_second": 14.397,
+      "eval_steps_per_second": 1.805,
+      "step": 560
+    },
+    {
+      "entropy": 0.6258987173438072,
+      "epoch": 1.3980699638118215,
+      "grad_norm": 0.45945441722869873,
+      "learning_rate": 7.194759147697293e-05,
+      "loss": 0.5705549716949463,
+      "mean_token_accuracy": 0.8329991281032563,
+      "num_tokens": 1415221.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3980699638118215,
+      "eval_entropy": 0.6500632069753797,
+      "eval_loss": 0.6059838533401489,
+      "eval_mean_token_accuracy": 0.8262855604793249,
+      "eval_num_tokens": 1415221.0,
+      "eval_runtime": 98.6469,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 580
+    },
+    {
+      "entropy": 0.6301489062607288,
+      "epoch": 1.4463208685162847,
+      "grad_norm": 0.5217350721359253,
+      "learning_rate": 7.18589177360681e-05,
+      "loss": 0.5778427124023438,
+      "mean_token_accuracy": 0.8305584967136384,
+      "num_tokens": 1463356.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4463208685162847,
+      "eval_entropy": 0.6478241879618569,
+      "eval_loss": 0.6014655828475952,
+      "eval_mean_token_accuracy": 0.8265940369514937,
+      "eval_num_tokens": 1463356.0,
+      "eval_runtime": 98.6523,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 600
+    },
+    {
+      "entropy": 0.6257719069719314,
+      "epoch": 1.494571773220748,
+      "grad_norm": 0.48310142755508423,
+      "learning_rate": 7.176013751259459e-05,
+      "loss": 0.5713310241699219,
+      "mean_token_accuracy": 0.8305533796548843,
+      "num_tokens": 1512340.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.494571773220748,
+      "eval_entropy": 0.6396401956510008,
+      "eval_loss": 0.5978795289993286,
+      "eval_mean_token_accuracy": 0.8273170630583603,
+      "eval_num_tokens": 1512340.0,
+      "eval_runtime": 98.4958,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 620
+    },
+    {
+      "entropy": 0.6162159442901611,
+      "epoch": 1.5428226779252112,
+      "grad_norm": 0.4500057101249695,
+      "learning_rate": 7.165127876018717e-05,
+      "loss": 0.565293264389038,
+      "mean_token_accuracy": 0.8311659932136536,
+      "num_tokens": 1560911.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5428226779252112,
+      "eval_entropy": 0.6381519051750054,
+      "eval_loss": 0.5963984131813049,
+      "eval_mean_token_accuracy": 0.8268409247478742,
+      "eval_num_tokens": 1560911.0,
+      "eval_runtime": 98.6803,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 640
+    },
+    {
+      "entropy": 0.6088747605681419,
+      "epoch": 1.5910735826296745,
+      "grad_norm": 0.46954667568206787,
+      "learning_rate": 7.153237228458511e-05,
+      "loss": 0.5602522850036621,
+      "mean_token_accuracy": 0.8345131903886795,
+      "num_tokens": 1609409.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5910735826296745,
+      "eval_entropy": 0.6413098875056492,
+      "eval_loss": 0.5938988327980042,
+      "eval_mean_token_accuracy": 0.828771137119679,
+      "eval_num_tokens": 1609409.0,
+      "eval_runtime": 98.4951,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 660
+    },
+    {
+      "entropy": 0.6209868296980858,
+      "epoch": 1.6393244873341375,
+      "grad_norm": 0.3867878317832947,
+      "learning_rate": 7.140345173491446e-05,
+      "loss": 0.5663797378540039,
+      "mean_token_accuracy": 0.8352224260568619,
+      "num_tokens": 1657430.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.6393244873341375,
+      "eval_entropy": 0.6182791748073664,
+      "eval_loss": 0.5919068455696106,
+      "eval_mean_token_accuracy": 0.8286421476455217,
+      "eval_num_tokens": 1657430.0,
+      "eval_runtime": 98.6211,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 680
+    },
+    {
+      "entropy": 0.6191923573613167,
+      "epoch": 1.6875753920386007,
+      "grad_norm": 0.40004226565361023,
+      "learning_rate": 7.12645535941658e-05,
+      "loss": 0.57517991065979,
+      "mean_token_accuracy": 0.8333286643028259,
+      "num_tokens": 1708058.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.6875753920386007,
+      "eval_entropy": 0.6244879199212856,
+      "eval_loss": 0.5897895693778992,
+      "eval_mean_token_accuracy": 0.8293621252092083,
+      "eval_num_tokens": 1708058.0,
+      "eval_runtime": 98.4974,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 700
+    },
+    {
+      "entropy": 0.6199282392859459,
+      "epoch": 1.7358262967430638,
+      "grad_norm": 0.36575791239738464,
+      "learning_rate": 7.111571716886997e-05,
+      "loss": 0.5717356681823731,
+      "mean_token_accuracy": 0.8321028783917427,
+      "num_tokens": 1761262.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.7358262967430638,
+      "eval_entropy": 0.6162440127201294,
+      "eval_loss": 0.5864423513412476,
+      "eval_mean_token_accuracy": 0.830874237786518,
+      "eval_num_tokens": 1761262.0,
+      "eval_runtime": 98.6109,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 720
+    },
+    {
+      "entropy": 0.6127455830574036,
+      "epoch": 1.784077201447527,
+      "grad_norm": 0.4778473973274231,
+      "learning_rate": 7.095698457797475e-05,
+      "loss": 0.5664590835571289,
+      "mean_token_accuracy": 0.8324130088090896,
+      "num_tokens": 1812127.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.784077201447527,
+      "eval_entropy": 0.6273092945640006,
+      "eval_loss": 0.5847376585006714,
+      "eval_mean_token_accuracy": 0.831051754817534,
+      "eval_num_tokens": 1812127.0,
+      "eval_runtime": 98.6513,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 740
+    },
+    {
+      "entropy": 0.6141144648194313,
+      "epoch": 1.8323281061519903,
+      "grad_norm": 0.6467089056968689,
+      "learning_rate": 7.078840074092574e-05,
+      "loss": 0.5548617362976074,
+      "mean_token_accuracy": 0.8342376738786698,
+      "num_tokens": 1858403.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.8323281061519903,
+      "eval_entropy": 0.5951976082967908,
+      "eval_loss": 0.5850171446800232,
+      "eval_mean_token_accuracy": 0.8303041870004675,
+      "eval_num_tokens": 1858403.0,
+      "eval_runtime": 98.6601,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 760
+    },
+    {
+      "entropy": 0.5961867064237595,
+      "epoch": 1.8805790108564535,
+      "grad_norm": 0.41280242800712585,
+      "learning_rate": 7.061001336495465e-05,
+      "loss": 0.5511267185211182,
+      "mean_token_accuracy": 0.8392465204000473,
+      "num_tokens": 1907361.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8805790108564535,
+      "eval_entropy": 0.6167017998320333,
+      "eval_loss": 0.5826791524887085,
+      "eval_mean_token_accuracy": 0.8309678050239434,
+      "eval_num_tokens": 1907361.0,
+      "eval_runtime": 98.6564,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 780
+    },
+    {
+      "entropy": 0.6025628983974457,
+      "epoch": 1.9288299155609168,
+      "grad_norm": 0.4078947901725769,
+      "learning_rate": 7.042187293157881e-05,
+      "loss": 0.5552026271820069,
+      "mean_token_accuracy": 0.8359778299927711,
+      "num_tokens": 1956605.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9288299155609168,
+      "eval_entropy": 0.6047008484601974,
+      "eval_loss": 0.5811123847961426,
+      "eval_mean_token_accuracy": 0.8320730804057603,
+      "eval_num_tokens": 1956605.0,
+      "eval_runtime": 98.6033,
+      "eval_samples_per_second": 14.401,
+      "eval_steps_per_second": 1.805,
+      "step": 800
+    },
+    {
+      "entropy": 0.6041995719075203,
+      "epoch": 1.97708082026538,
+      "grad_norm": 0.543890118598938,
+      "learning_rate": 7.022403268231535e-05,
+      "loss": 0.554384422302246,
+      "mean_token_accuracy": 0.8380729153752327,
+      "num_tokens": 2003308.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.97708082026538,
+      "eval_entropy": 0.6005127917179901,
+      "eval_loss": 0.5800131559371948,
+      "eval_mean_token_accuracy": 0.8313186547059691,
+      "eval_num_tokens": 2003308.0,
+      "eval_runtime": 98.4119,
+      "eval_samples_per_second": 14.429,
+      "eval_steps_per_second": 1.809,
+      "step": 820
+    },
+    {
+      "entropy": 0.567442889396961,
+      "epoch": 2.0241254523522314,
+      "grad_norm": 0.5015465021133423,
+      "learning_rate": 7.001654860361456e-05,
+      "loss": 0.5208251476287842,
+      "mean_token_accuracy": 0.8442784960453327,
+      "num_tokens": 2051331.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0241254523522314,
+      "eval_entropy": 0.5785037809878253,
+      "eval_loss": 0.5865591168403625,
+      "eval_mean_token_accuracy": 0.8307993857378371,
+      "eval_num_tokens": 2051331.0,
+      "eval_runtime": 98.5061,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 840
+    },
+    {
+      "entropy": 0.5552012033760547,
+      "epoch": 2.0723763570566947,
+      "grad_norm": 0.5975943207740784,
+      "learning_rate": 6.979947941101646e-05,
+      "loss": 0.5064489841461182,
+      "mean_token_accuracy": 0.8468396887183189,
+      "num_tokens": 2101528.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0723763570566947,
+      "eval_entropy": 0.5844205239515626,
+      "eval_loss": 0.585284948348999,
+      "eval_mean_token_accuracy": 0.8309797065311604,
+      "eval_num_tokens": 2101528.0,
+      "eval_runtime": 98.4982,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 860
+    },
+    {
+      "entropy": 0.5401638634502888,
+      "epoch": 2.120627261761158,
+      "grad_norm": 0.4194582998752594,
+      "learning_rate": 6.957288653253486e-05,
+      "loss": 0.49037814140319824,
+      "mean_token_accuracy": 0.8528496414422989,
+      "num_tokens": 2149550.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.120627261761158,
+      "eval_entropy": 0.574507936835289,
+      "eval_loss": 0.5836747288703918,
+      "eval_mean_token_accuracy": 0.8325241643391298,
+      "eval_num_tokens": 2149550.0,
+      "eval_runtime": 98.5592,
+      "eval_samples_per_second": 14.408,
+      "eval_steps_per_second": 1.806,
+      "step": 880
+    },
+    {
+      "entropy": 0.5606910541653634,
+      "epoch": 2.168878166465621,
+      "grad_norm": 0.5395355820655823,
+      "learning_rate": 6.933683409127401e-05,
+      "loss": 0.5089571475982666,
+      "mean_token_accuracy": 0.8459660410881042,
+      "num_tokens": 2202162.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.168878166465621,
+      "eval_entropy": 0.5779740023479033,
+      "eval_loss": 0.5816043615341187,
+      "eval_mean_token_accuracy": 0.8326318036304431,
+      "eval_num_tokens": 2202162.0,
+      "eval_runtime": 98.4386,
+      "eval_samples_per_second": 14.425,
+      "eval_steps_per_second": 1.808,
+      "step": 900
+    },
+    {
+      "entropy": 0.5380903735756875,
+      "epoch": 2.2171290711700844,
+      "grad_norm": 0.46588417887687683,
+      "learning_rate": 6.909138888728257e-05,
+      "loss": 0.4902662754058838,
+      "mean_token_accuracy": 0.8493925213813782,
+      "num_tokens": 2256539.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2171290711700844,
+      "eval_entropy": 0.570154261220707,
+      "eval_loss": 0.5827520489692688,
+      "eval_mean_token_accuracy": 0.8321914167216654,
+      "eval_num_tokens": 2256539.0,
+      "eval_runtime": 98.5674,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 920
+    },
+    {
+      "entropy": 0.558555257320404,
+      "epoch": 2.2653799758745476,
+      "grad_norm": 0.44576048851013184,
+      "learning_rate": 6.883662037864996e-05,
+      "loss": 0.5000699520111084,
+      "mean_token_accuracy": 0.8475695163011551,
+      "num_tokens": 2299698.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2653799758745476,
+      "eval_entropy": 0.5715417749761196,
+      "eval_loss": 0.5831062197685242,
+      "eval_mean_token_accuracy": 0.8327409899636601,
+      "eval_num_tokens": 2299698.0,
+      "eval_runtime": 98.568,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 940
+    },
+    {
+      "entropy": 0.5582522705197335,
+      "epoch": 2.313630880579011,
+      "grad_norm": 0.38966405391693115,
+      "learning_rate": 6.857260066185045e-05,
+      "loss": 0.5132341861724854,
+      "mean_token_accuracy": 0.8466864034533501,
+      "num_tokens": 2347134.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.313630880579011,
+      "eval_entropy": 0.5823918579334624,
+      "eval_loss": 0.5797588229179382,
+      "eval_mean_token_accuracy": 0.8326037566313583,
+      "eval_num_tokens": 2347134.0,
+      "eval_runtime": 98.6222,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 960
+    },
+    {
+      "entropy": 0.5557353265583516,
+      "epoch": 2.361881785283474,
+      "grad_norm": 0.42300811409950256,
+      "learning_rate": 6.829940445134089e-05,
+      "loss": 0.5035375118255615,
+      "mean_token_accuracy": 0.8479988545179367,
+      "num_tokens": 2394568.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.361881785283474,
+      "eval_entropy": 0.5542296482605881,
+      "eval_loss": 0.5800604820251465,
+      "eval_mean_token_accuracy": 0.8336404163515969,
+      "eval_num_tokens": 2394568.0,
+      "eval_runtime": 98.615,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 980
+    },
+    {
+      "entropy": 0.5492161035537719,
+      "epoch": 2.4101326899879374,
+      "grad_norm": 0.47482752799987793,
+      "learning_rate": 6.801710905841724e-05,
+      "loss": 0.5032052040100098,
+      "mean_token_accuracy": 0.8488596931099892,
+      "num_tokens": 2443253.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4101326899879374,
+      "eval_entropy": 0.5755258423558781,
+      "eval_loss": 0.5764774680137634,
+      "eval_mean_token_accuracy": 0.8336571763070781,
+      "eval_num_tokens": 2443253.0,
+      "eval_runtime": 98.4765,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.0729150094336e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1020/trainer_state.json

@@ -0,0 +1,1105 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.4583835946924006,
+  "eval_steps": 20,
+  "global_step": 1020,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    },
+    {
+      "entropy": 0.8518766984343529,
+      "epoch": 0.28950542822677927,
+      "grad_norm": 0.635079026222229,
+      "learning_rate": 2.0729210702058663e-05,
+      "loss": 0.8058656692504883,
+      "mean_token_accuracy": 0.7847469851374627,
+      "num_tokens": 298041.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28950542822677927,
+      "eval_entropy": 0.838603059227547,
+      "eval_loss": 0.7992516756057739,
+      "eval_mean_token_accuracy": 0.7849764954508021,
+      "eval_num_tokens": 298041.0,
+      "eval_runtime": 98.5627,
+      "eval_samples_per_second": 14.407,
+      "eval_steps_per_second": 1.806,
+      "step": 120
+    },
+    {
+      "entropy": 0.8069365546107292,
+      "epoch": 0.33775633293124246,
+      "grad_norm": 0.5280472636222839,
+      "learning_rate": 2.421311166038785e-05,
+      "loss": 0.7562496662139893,
+      "mean_token_accuracy": 0.7897997036576271,
+      "num_tokens": 348955.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33775633293124246,
+      "eval_entropy": 0.8161682168419442,
+      "eval_loss": 0.7599642276763916,
+      "eval_mean_token_accuracy": 0.7912511644738444,
+      "eval_num_tokens": 348955.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 140
+    },
+    {
+      "entropy": 0.7952415809035301,
+      "epoch": 0.38600723763570566,
+      "grad_norm": 0.5387083888053894,
+      "learning_rate": 2.7697012618717034e-05,
+      "loss": 0.740808629989624,
+      "mean_token_accuracy": 0.7941806614398956,
+      "num_tokens": 395798.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38600723763570566,
+      "eval_entropy": 0.7856516191798649,
+      "eval_loss": 0.7324908971786499,
+      "eval_mean_token_accuracy": 0.7980317825011993,
+      "eval_num_tokens": 395798.0,
+      "eval_runtime": 98.6378,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 160
+    },
+    {
+      "entropy": 0.7537026956677437,
+      "epoch": 0.43425814234016885,
+      "grad_norm": 0.5521425604820251,
+      "learning_rate": 3.118091357704622e-05,
+      "loss": 0.7073616027832031,
+      "mean_token_accuracy": 0.8006261065602303,
+      "num_tokens": 446437.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43425814234016885,
+      "eval_entropy": 0.7470693678668375,
+      "eval_loss": 0.711143434047699,
+      "eval_mean_token_accuracy": 0.801459076029531,
+      "eval_num_tokens": 446437.0,
+      "eval_runtime": 98.6667,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 180
+    },
+    {
+      "entropy": 0.7328549668192863,
+      "epoch": 0.4825090470446321,
+      "grad_norm": 0.4901895821094513,
+      "learning_rate": 3.4664814535375406e-05,
+      "loss": 0.6874266624450683,
+      "mean_token_accuracy": 0.8066758885979652,
+      "num_tokens": 497682.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4825090470446321,
+      "eval_entropy": 0.7374835154983435,
+      "eval_loss": 0.6967977285385132,
+      "eval_mean_token_accuracy": 0.8055616520094068,
+      "eval_num_tokens": 497682.0,
+      "eval_runtime": 98.6234,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 200
+    },
+    {
+      "entropy": 0.7372006759047508,
+      "epoch": 0.5307599517490953,
+      "grad_norm": 0.5516473650932312,
+      "learning_rate": 3.814871549370459e-05,
+      "loss": 0.6906380653381348,
+      "mean_token_accuracy": 0.8057604253292083,
+      "num_tokens": 544670.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5307599517490953,
+      "eval_entropy": 0.710068748573239,
+      "eval_loss": 0.6860950589179993,
+      "eval_mean_token_accuracy": 0.8069355440943429,
+      "eval_num_tokens": 544670.0,
+      "eval_runtime": 98.4618,
+      "eval_samples_per_second": 14.422,
+      "eval_steps_per_second": 1.808,
+      "step": 220
+    },
+    {
+      "entropy": 0.7099859565496445,
+      "epoch": 0.5790108564535585,
+      "grad_norm": 0.457378625869751,
+      "learning_rate": 4.1632616452033785e-05,
+      "loss": 0.6674468517303467,
+      "mean_token_accuracy": 0.8087703332304954,
+      "num_tokens": 592926.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5790108564535585,
+      "eval_entropy": 0.717441116156203,
+      "eval_loss": 0.6765118837356567,
+      "eval_mean_token_accuracy": 0.8095786246690857,
+      "eval_num_tokens": 592926.0,
+      "eval_runtime": 98.6828,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 240
+    },
+    {
+      "entropy": 0.7170855522155761,
+      "epoch": 0.6272617611580217,
+      "grad_norm": 0.4604854881763458,
+      "learning_rate": 4.5116517410362974e-05,
+      "loss": 0.6647133827209473,
+      "mean_token_accuracy": 0.8115006521344185,
+      "num_tokens": 639077.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6272617611580217,
+      "eval_entropy": 0.7082702219486237,
+      "eval_loss": 0.6722468733787537,
+      "eval_mean_token_accuracy": 0.8100237367528208,
+      "eval_num_tokens": 639077.0,
+      "eval_runtime": 98.6242,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 260
+    },
+    {
+      "entropy": 0.691286925971508,
+      "epoch": 0.6755126658624849,
+      "grad_norm": 0.4626481831073761,
+      "learning_rate": 4.860041836869215e-05,
+      "loss": 0.6543298721313476,
+      "mean_token_accuracy": 0.8142751976847649,
+      "num_tokens": 691028.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6755126658624849,
+      "eval_entropy": 0.7341568747932992,
+      "eval_loss": 0.658997654914856,
+      "eval_mean_token_accuracy": 0.813439255684949,
+      "eval_num_tokens": 691028.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 280
+    },
+    {
+      "entropy": 0.691513468325138,
+      "epoch": 0.7237635705669482,
+      "grad_norm": 0.5434145331382751,
+      "learning_rate": 5.2084319327021346e-05,
+      "loss": 0.6393396377563476,
+      "mean_token_accuracy": 0.8172704800963402,
+      "num_tokens": 740255.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7237635705669482,
+      "eval_entropy": 0.699160492487168,
+      "eval_loss": 0.6522852182388306,
+      "eval_mean_token_accuracy": 0.8143363323774231,
+      "eval_num_tokens": 740255.0,
+      "eval_runtime": 98.6233,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 300
+    },
+    {
+      "entropy": 0.6918429866433143,
+      "epoch": 0.7720144752714113,
+      "grad_norm": 0.47131890058517456,
+      "learning_rate": 5.556822028535052e-05,
+      "loss": 0.6503581523895263,
+      "mean_token_accuracy": 0.8149650946259499,
+      "num_tokens": 789529.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7720144752714113,
+      "eval_entropy": 0.6949931701247611,
+      "eval_loss": 0.6464414596557617,
+      "eval_mean_token_accuracy": 0.816310279489903,
+      "eval_num_tokens": 789529.0,
+      "eval_runtime": 98.4759,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 320
+    },
+    {
+      "entropy": 0.6926439955830574,
+      "epoch": 0.8202653799758746,
+      "grad_norm": 0.46790942549705505,
+      "learning_rate": 5.905212124367972e-05,
+      "loss": 0.6453651428222656,
+      "mean_token_accuracy": 0.8171535924077034,
+      "num_tokens": 841584.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8202653799758746,
+      "eval_entropy": 0.6810062698433909,
+      "eval_loss": 0.6440736651420593,
+      "eval_mean_token_accuracy": 0.8166696479481258,
+      "eval_num_tokens": 841584.0,
+      "eval_runtime": 98.6479,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 340
+    },
+    {
+      "entropy": 0.7038845136761666,
+      "epoch": 0.8685162846803377,
+      "grad_norm": 0.5038695335388184,
+      "learning_rate": 6.253602220200891e-05,
+      "loss": 0.6558614253997803,
+      "mean_token_accuracy": 0.8127013072371483,
+      "num_tokens": 885769.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8685162846803377,
+      "eval_entropy": 0.6752923710292644,
+      "eval_loss": 0.6410524845123291,
+      "eval_mean_token_accuracy": 0.817534717950928,
+      "eval_num_tokens": 885769.0,
+      "eval_runtime": 98.5986,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 360
+    },
+    {
+      "entropy": 0.6897840186953544,
+      "epoch": 0.916767189384801,
+      "grad_norm": 0.5168097615242004,
+      "learning_rate": 6.601992316033808e-05,
+      "loss": 0.6328047752380371,
+      "mean_token_accuracy": 0.8181755512952804,
+      "num_tokens": 930056.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.916767189384801,
+      "eval_entropy": 0.6816390936964014,
+      "eval_loss": 0.6402509212493896,
+      "eval_mean_token_accuracy": 0.8172605586855599,
+      "eval_num_tokens": 930056.0,
+      "eval_runtime": 98.4787,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 380
+    },
+    {
+      "entropy": 0.6798740655183793,
+      "epoch": 0.9650180940892642,
+      "grad_norm": 0.4609147906303406,
+      "learning_rate": 6.950382411866727e-05,
+      "loss": 0.6358067989349365,
+      "mean_token_accuracy": 0.8193414479494094,
+      "num_tokens": 976892.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9650180940892642,
+      "eval_entropy": 0.6841439577970612,
+      "eval_loss": 0.6335230469703674,
+      "eval_mean_token_accuracy": 0.8190797965848045,
+      "eval_num_tokens": 976892.0,
+      "eval_runtime": 98.5983,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 400
+    },
+    {
+      "entropy": 0.6647876103719076,
+      "epoch": 1.0120627261761157,
+      "grad_norm": 0.46518462896347046,
+      "learning_rate": 7.229074030587624e-05,
+      "loss": 0.6112009525299072,
+      "mean_token_accuracy": 0.8212849283829714,
+      "num_tokens": 1024472.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0120627261761157,
+      "eval_entropy": 0.6612423688508151,
+      "eval_loss": 0.6311497688293457,
+      "eval_mean_token_accuracy": 0.8194578638237514,
+      "eval_num_tokens": 1024472.0,
+      "eval_runtime": 98.4872,
+      "eval_samples_per_second": 14.418,
+      "eval_steps_per_second": 1.807,
+      "step": 420
+    },
+    {
+      "entropy": 0.6288895070552826,
+      "epoch": 1.060313630880579,
+      "grad_norm": 0.4342021644115448,
+      "learning_rate": 7.228358026812833e-05,
+      "loss": 0.5853999614715576,
+      "mean_token_accuracy": 0.82901301831007,
+      "num_tokens": 1076964.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060313630880579,
+      "eval_entropy": 0.6567474329069759,
+      "eval_loss": 0.6245927810668945,
+      "eval_mean_token_accuracy": 0.8211770050981072,
+      "eval_num_tokens": 1076964.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 440
+    },
+    {
+      "entropy": 0.639747454226017,
+      "epoch": 1.1085645355850422,
+      "grad_norm": 0.5338298082351685,
+      "learning_rate": 7.226619357333845e-05,
+      "loss": 0.5847910404205322,
+      "mean_token_accuracy": 0.8279331669211387,
+      "num_tokens": 1126064.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1085645355850422,
+      "eval_entropy": 0.6399250211340658,
+      "eval_loss": 0.6245712637901306,
+      "eval_mean_token_accuracy": 0.8209733875949731,
+      "eval_num_tokens": 1126064.0,
+      "eval_runtime": 98.6581,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 460
+    },
+    {
+      "entropy": 0.6248803421854973,
+      "epoch": 1.1568154402895054,
+      "grad_norm": 0.4402505159378052,
+      "learning_rate": 7.223858514173552e-05,
+      "loss": 0.5847790241241455,
+      "mean_token_accuracy": 0.8273933246731758,
+      "num_tokens": 1176912.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1568154402895054,
+      "eval_entropy": 0.6459465750147787,
+      "eval_loss": 0.615716278553009,
+      "eval_mean_token_accuracy": 0.8228987813665626,
+      "eval_num_tokens": 1176912.0,
+      "eval_runtime": 98.5987,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 480
+    },
+    {
+      "entropy": 0.6356995061039925,
+      "epoch": 1.2050663449939687,
+      "grad_norm": 0.5318694114685059,
+      "learning_rate": 7.220076278617914e-05,
+      "loss": 0.5902805328369141,
+      "mean_token_accuracy": 0.8279137581586837,
+      "num_tokens": 1223383.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2050663449939687,
+      "eval_entropy": 0.6509443488013879,
+      "eval_loss": 0.6174858808517456,
+      "eval_mean_token_accuracy": 0.8229377031996009,
+      "eval_num_tokens": 1223383.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 500
+    },
+    {
+      "entropy": 0.6378119193017483,
+      "epoch": 1.2533172496984317,
+      "grad_norm": 0.5385730266571045,
+      "learning_rate": 7.215273720994853e-05,
+      "loss": 0.5850183486938476,
+      "mean_token_accuracy": 0.8282771646976471,
+      "num_tokens": 1273534.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2533172496984317,
+      "eval_entropy": 0.6438969477508845,
+      "eval_loss": 0.6134508848190308,
+      "eval_mean_token_accuracy": 0.823984161186754,
+      "eval_num_tokens": 1273534.0,
+      "eval_runtime": 98.4814,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 520
+    },
+    {
+      "entropy": 0.6330174207687378,
+      "epoch": 1.301568154402895,
+      "grad_norm": 0.4950766861438751,
+      "learning_rate": 7.209452200371367e-05,
+      "loss": 0.5886627674102783,
+      "mean_token_accuracy": 0.828557687997818,
+      "num_tokens": 1323351.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.301568154402895,
+      "eval_entropy": 0.6342737025089478,
+      "eval_loss": 0.611145555973053,
+      "eval_mean_token_accuracy": 0.8250270244780551,
+      "eval_num_tokens": 1323351.0,
+      "eval_runtime": 98.6698,
+      "eval_samples_per_second": 14.391,
+      "eval_steps_per_second": 1.804,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270076125860214,
+      "epoch": 1.3498190591073582,
+      "grad_norm": 0.435604989528656,
+      "learning_rate": 7.20261336416893e-05,
+      "loss": 0.5834732532501221,
+      "mean_token_accuracy": 0.8290600389242172,
+      "num_tokens": 1371508.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3498190591073582,
+      "eval_entropy": 0.6624904408883513,
+      "eval_loss": 0.6063718795776367,
+      "eval_mean_token_accuracy": 0.825260305002834,
+      "eval_num_tokens": 1371508.0,
+      "eval_runtime": 98.6348,
+      "eval_samples_per_second": 14.397,
+      "eval_steps_per_second": 1.805,
+      "step": 560
+    },
+    {
+      "entropy": 0.6258987173438072,
+      "epoch": 1.3980699638118215,
+      "grad_norm": 0.45945441722869873,
+      "learning_rate": 7.194759147697293e-05,
+      "loss": 0.5705549716949463,
+      "mean_token_accuracy": 0.8329991281032563,
+      "num_tokens": 1415221.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3980699638118215,
+      "eval_entropy": 0.6500632069753797,
+      "eval_loss": 0.6059838533401489,
+      "eval_mean_token_accuracy": 0.8262855604793249,
+      "eval_num_tokens": 1415221.0,
+      "eval_runtime": 98.6469,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 580
+    },
+    {
+      "entropy": 0.6301489062607288,
+      "epoch": 1.4463208685162847,
+      "grad_norm": 0.5217350721359253,
+      "learning_rate": 7.18589177360681e-05,
+      "loss": 0.5778427124023438,
+      "mean_token_accuracy": 0.8305584967136384,
+      "num_tokens": 1463356.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4463208685162847,
+      "eval_entropy": 0.6478241879618569,
+      "eval_loss": 0.6014655828475952,
+      "eval_mean_token_accuracy": 0.8265940369514937,
+      "eval_num_tokens": 1463356.0,
+      "eval_runtime": 98.6523,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 600
+    },
+    {
+      "entropy": 0.6257719069719314,
+      "epoch": 1.494571773220748,
+      "grad_norm": 0.48310142755508423,
+      "learning_rate": 7.176013751259459e-05,
+      "loss": 0.5713310241699219,
+      "mean_token_accuracy": 0.8305533796548843,
+      "num_tokens": 1512340.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.494571773220748,
+      "eval_entropy": 0.6396401956510008,
+      "eval_loss": 0.5978795289993286,
+      "eval_mean_token_accuracy": 0.8273170630583603,
+      "eval_num_tokens": 1512340.0,
+      "eval_runtime": 98.4958,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 620
+    },
+    {
+      "entropy": 0.6162159442901611,
+      "epoch": 1.5428226779252112,
+      "grad_norm": 0.4500057101249695,
+      "learning_rate": 7.165127876018717e-05,
+      "loss": 0.565293264389038,
+      "mean_token_accuracy": 0.8311659932136536,
+      "num_tokens": 1560911.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5428226779252112,
+      "eval_entropy": 0.6381519051750054,
+      "eval_loss": 0.5963984131813049,
+      "eval_mean_token_accuracy": 0.8268409247478742,
+      "eval_num_tokens": 1560911.0,
+      "eval_runtime": 98.6803,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 640
+    },
+    {
+      "entropy": 0.6088747605681419,
+      "epoch": 1.5910735826296745,
+      "grad_norm": 0.46954667568206787,
+      "learning_rate": 7.153237228458511e-05,
+      "loss": 0.5602522850036621,
+      "mean_token_accuracy": 0.8345131903886795,
+      "num_tokens": 1609409.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5910735826296745,
+      "eval_entropy": 0.6413098875056492,
+      "eval_loss": 0.5938988327980042,
+      "eval_mean_token_accuracy": 0.828771137119679,
+      "eval_num_tokens": 1609409.0,
+      "eval_runtime": 98.4951,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 660
+    },
+    {
+      "entropy": 0.6209868296980858,
+      "epoch": 1.6393244873341375,
+      "grad_norm": 0.3867878317832947,
+      "learning_rate": 7.140345173491446e-05,
+      "loss": 0.5663797378540039,
+      "mean_token_accuracy": 0.8352224260568619,
+      "num_tokens": 1657430.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.6393244873341375,
+      "eval_entropy": 0.6182791748073664,
+      "eval_loss": 0.5919068455696106,
+      "eval_mean_token_accuracy": 0.8286421476455217,
+      "eval_num_tokens": 1657430.0,
+      "eval_runtime": 98.6211,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 680
+    },
+    {
+      "entropy": 0.6191923573613167,
+      "epoch": 1.6875753920386007,
+      "grad_norm": 0.40004226565361023,
+      "learning_rate": 7.12645535941658e-05,
+      "loss": 0.57517991065979,
+      "mean_token_accuracy": 0.8333286643028259,
+      "num_tokens": 1708058.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.6875753920386007,
+      "eval_entropy": 0.6244879199212856,
+      "eval_loss": 0.5897895693778992,
+      "eval_mean_token_accuracy": 0.8293621252092083,
+      "eval_num_tokens": 1708058.0,
+      "eval_runtime": 98.4974,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 700
+    },
+    {
+      "entropy": 0.6199282392859459,
+      "epoch": 1.7358262967430638,
+      "grad_norm": 0.36575791239738464,
+      "learning_rate": 7.111571716886997e-05,
+      "loss": 0.5717356681823731,
+      "mean_token_accuracy": 0.8321028783917427,
+      "num_tokens": 1761262.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.7358262967430638,
+      "eval_entropy": 0.6162440127201294,
+      "eval_loss": 0.5864423513412476,
+      "eval_mean_token_accuracy": 0.830874237786518,
+      "eval_num_tokens": 1761262.0,
+      "eval_runtime": 98.6109,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 720
+    },
+    {
+      "entropy": 0.6127455830574036,
+      "epoch": 1.784077201447527,
+      "grad_norm": 0.4778473973274231,
+      "learning_rate": 7.095698457797475e-05,
+      "loss": 0.5664590835571289,
+      "mean_token_accuracy": 0.8324130088090896,
+      "num_tokens": 1812127.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.784077201447527,
+      "eval_entropy": 0.6273092945640006,
+      "eval_loss": 0.5847376585006714,
+      "eval_mean_token_accuracy": 0.831051754817534,
+      "eval_num_tokens": 1812127.0,
+      "eval_runtime": 98.6513,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 740
+    },
+    {
+      "entropy": 0.6141144648194313,
+      "epoch": 1.8323281061519903,
+      "grad_norm": 0.6467089056968689,
+      "learning_rate": 7.078840074092574e-05,
+      "loss": 0.5548617362976074,
+      "mean_token_accuracy": 0.8342376738786698,
+      "num_tokens": 1858403.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.8323281061519903,
+      "eval_entropy": 0.5951976082967908,
+      "eval_loss": 0.5850171446800232,
+      "eval_mean_token_accuracy": 0.8303041870004675,
+      "eval_num_tokens": 1858403.0,
+      "eval_runtime": 98.6601,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 760
+    },
+    {
+      "entropy": 0.5961867064237595,
+      "epoch": 1.8805790108564535,
+      "grad_norm": 0.41280242800712585,
+      "learning_rate": 7.061001336495465e-05,
+      "loss": 0.5511267185211182,
+      "mean_token_accuracy": 0.8392465204000473,
+      "num_tokens": 1907361.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8805790108564535,
+      "eval_entropy": 0.6167017998320333,
+      "eval_loss": 0.5826791524887085,
+      "eval_mean_token_accuracy": 0.8309678050239434,
+      "eval_num_tokens": 1907361.0,
+      "eval_runtime": 98.6564,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 780
+    },
+    {
+      "entropy": 0.6025628983974457,
+      "epoch": 1.9288299155609168,
+      "grad_norm": 0.4078947901725769,
+      "learning_rate": 7.042187293157881e-05,
+      "loss": 0.5552026271820069,
+      "mean_token_accuracy": 0.8359778299927711,
+      "num_tokens": 1956605.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9288299155609168,
+      "eval_entropy": 0.6047008484601974,
+      "eval_loss": 0.5811123847961426,
+      "eval_mean_token_accuracy": 0.8320730804057603,
+      "eval_num_tokens": 1956605.0,
+      "eval_runtime": 98.6033,
+      "eval_samples_per_second": 14.401,
+      "eval_steps_per_second": 1.805,
+      "step": 800
+    },
+    {
+      "entropy": 0.6041995719075203,
+      "epoch": 1.97708082026538,
+      "grad_norm": 0.543890118598938,
+      "learning_rate": 7.022403268231535e-05,
+      "loss": 0.554384422302246,
+      "mean_token_accuracy": 0.8380729153752327,
+      "num_tokens": 2003308.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.97708082026538,
+      "eval_entropy": 0.6005127917179901,
+      "eval_loss": 0.5800131559371948,
+      "eval_mean_token_accuracy": 0.8313186547059691,
+      "eval_num_tokens": 2003308.0,
+      "eval_runtime": 98.4119,
+      "eval_samples_per_second": 14.429,
+      "eval_steps_per_second": 1.809,
+      "step": 820
+    },
+    {
+      "entropy": 0.567442889396961,
+      "epoch": 2.0241254523522314,
+      "grad_norm": 0.5015465021133423,
+      "learning_rate": 7.001654860361456e-05,
+      "loss": 0.5208251476287842,
+      "mean_token_accuracy": 0.8442784960453327,
+      "num_tokens": 2051331.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0241254523522314,
+      "eval_entropy": 0.5785037809878253,
+      "eval_loss": 0.5865591168403625,
+      "eval_mean_token_accuracy": 0.8307993857378371,
+      "eval_num_tokens": 2051331.0,
+      "eval_runtime": 98.5061,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 840
+    },
+    {
+      "entropy": 0.5552012033760547,
+      "epoch": 2.0723763570566947,
+      "grad_norm": 0.5975943207740784,
+      "learning_rate": 6.979947941101646e-05,
+      "loss": 0.5064489841461182,
+      "mean_token_accuracy": 0.8468396887183189,
+      "num_tokens": 2101528.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0723763570566947,
+      "eval_entropy": 0.5844205239515626,
+      "eval_loss": 0.585284948348999,
+      "eval_mean_token_accuracy": 0.8309797065311604,
+      "eval_num_tokens": 2101528.0,
+      "eval_runtime": 98.4982,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 860
+    },
+    {
+      "entropy": 0.5401638634502888,
+      "epoch": 2.120627261761158,
+      "grad_norm": 0.4194582998752594,
+      "learning_rate": 6.957288653253486e-05,
+      "loss": 0.49037814140319824,
+      "mean_token_accuracy": 0.8528496414422989,
+      "num_tokens": 2149550.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.120627261761158,
+      "eval_entropy": 0.574507936835289,
+      "eval_loss": 0.5836747288703918,
+      "eval_mean_token_accuracy": 0.8325241643391298,
+      "eval_num_tokens": 2149550.0,
+      "eval_runtime": 98.5592,
+      "eval_samples_per_second": 14.408,
+      "eval_steps_per_second": 1.806,
+      "step": 880
+    },
+    {
+      "entropy": 0.5606910541653634,
+      "epoch": 2.168878166465621,
+      "grad_norm": 0.5395355820655823,
+      "learning_rate": 6.933683409127401e-05,
+      "loss": 0.5089571475982666,
+      "mean_token_accuracy": 0.8459660410881042,
+      "num_tokens": 2202162.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.168878166465621,
+      "eval_entropy": 0.5779740023479033,
+      "eval_loss": 0.5816043615341187,
+      "eval_mean_token_accuracy": 0.8326318036304431,
+      "eval_num_tokens": 2202162.0,
+      "eval_runtime": 98.4386,
+      "eval_samples_per_second": 14.425,
+      "eval_steps_per_second": 1.808,
+      "step": 900
+    },
+    {
+      "entropy": 0.5380903735756875,
+      "epoch": 2.2171290711700844,
+      "grad_norm": 0.46588417887687683,
+      "learning_rate": 6.909138888728257e-05,
+      "loss": 0.4902662754058838,
+      "mean_token_accuracy": 0.8493925213813782,
+      "num_tokens": 2256539.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2171290711700844,
+      "eval_entropy": 0.570154261220707,
+      "eval_loss": 0.5827520489692688,
+      "eval_mean_token_accuracy": 0.8321914167216654,
+      "eval_num_tokens": 2256539.0,
+      "eval_runtime": 98.5674,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 920
+    },
+    {
+      "entropy": 0.558555257320404,
+      "epoch": 2.2653799758745476,
+      "grad_norm": 0.44576048851013184,
+      "learning_rate": 6.883662037864996e-05,
+      "loss": 0.5000699520111084,
+      "mean_token_accuracy": 0.8475695163011551,
+      "num_tokens": 2299698.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2653799758745476,
+      "eval_entropy": 0.5715417749761196,
+      "eval_loss": 0.5831062197685242,
+      "eval_mean_token_accuracy": 0.8327409899636601,
+      "eval_num_tokens": 2299698.0,
+      "eval_runtime": 98.568,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 940
+    },
+    {
+      "entropy": 0.5582522705197335,
+      "epoch": 2.313630880579011,
+      "grad_norm": 0.38966405391693115,
+      "learning_rate": 6.857260066185045e-05,
+      "loss": 0.5132341861724854,
+      "mean_token_accuracy": 0.8466864034533501,
+      "num_tokens": 2347134.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.313630880579011,
+      "eval_entropy": 0.5823918579334624,
+      "eval_loss": 0.5797588229179382,
+      "eval_mean_token_accuracy": 0.8326037566313583,
+      "eval_num_tokens": 2347134.0,
+      "eval_runtime": 98.6222,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 960
+    },
+    {
+      "entropy": 0.5557353265583516,
+      "epoch": 2.361881785283474,
+      "grad_norm": 0.42300811409950256,
+      "learning_rate": 6.829940445134089e-05,
+      "loss": 0.5035375118255615,
+      "mean_token_accuracy": 0.8479988545179367,
+      "num_tokens": 2394568.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.361881785283474,
+      "eval_entropy": 0.5542296482605881,
+      "eval_loss": 0.5800604820251465,
+      "eval_mean_token_accuracy": 0.8336404163515969,
+      "eval_num_tokens": 2394568.0,
+      "eval_runtime": 98.615,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 980
+    },
+    {
+      "entropy": 0.5492161035537719,
+      "epoch": 2.4101326899879374,
+      "grad_norm": 0.47482752799987793,
+      "learning_rate": 6.801710905841724e-05,
+      "loss": 0.5032052040100098,
+      "mean_token_accuracy": 0.8488596931099892,
+      "num_tokens": 2443253.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4101326899879374,
+      "eval_entropy": 0.5755258423558781,
+      "eval_loss": 0.5764774680137634,
+      "eval_mean_token_accuracy": 0.8336571763070781,
+      "eval_num_tokens": 2443253.0,
+      "eval_runtime": 98.4765,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5439704649150372,
+      "epoch": 2.4583835946924006,
+      "grad_norm": 0.3656543493270874,
+      "learning_rate": 6.772579436933644e-05,
+      "loss": 0.49903340339660646,
+      "mean_token_accuracy": 0.8499931409955025,
+      "num_tokens": 2495178.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4583835946924006,
+      "eval_entropy": 0.5712662409530597,
+      "eval_loss": 0.5777108073234558,
+      "eval_mean_token_accuracy": 0.8336941352051296,
+      "eval_num_tokens": 2495178.0,
+      "eval_runtime": 98.678,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 1020
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.17605631420416e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1040/trainer_state.json

@@ -0,0 +1,1126 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.5066344993968634,
+  "eval_steps": 20,
+  "global_step": 1040,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    },
+    {
+      "entropy": 0.8518766984343529,
+      "epoch": 0.28950542822677927,
+      "grad_norm": 0.635079026222229,
+      "learning_rate": 2.0729210702058663e-05,
+      "loss": 0.8058656692504883,
+      "mean_token_accuracy": 0.7847469851374627,
+      "num_tokens": 298041.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28950542822677927,
+      "eval_entropy": 0.838603059227547,
+      "eval_loss": 0.7992516756057739,
+      "eval_mean_token_accuracy": 0.7849764954508021,
+      "eval_num_tokens": 298041.0,
+      "eval_runtime": 98.5627,
+      "eval_samples_per_second": 14.407,
+      "eval_steps_per_second": 1.806,
+      "step": 120
+    },
+    {
+      "entropy": 0.8069365546107292,
+      "epoch": 0.33775633293124246,
+      "grad_norm": 0.5280472636222839,
+      "learning_rate": 2.421311166038785e-05,
+      "loss": 0.7562496662139893,
+      "mean_token_accuracy": 0.7897997036576271,
+      "num_tokens": 348955.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33775633293124246,
+      "eval_entropy": 0.8161682168419442,
+      "eval_loss": 0.7599642276763916,
+      "eval_mean_token_accuracy": 0.7912511644738444,
+      "eval_num_tokens": 348955.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 140
+    },
+    {
+      "entropy": 0.7952415809035301,
+      "epoch": 0.38600723763570566,
+      "grad_norm": 0.5387083888053894,
+      "learning_rate": 2.7697012618717034e-05,
+      "loss": 0.740808629989624,
+      "mean_token_accuracy": 0.7941806614398956,
+      "num_tokens": 395798.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38600723763570566,
+      "eval_entropy": 0.7856516191798649,
+      "eval_loss": 0.7324908971786499,
+      "eval_mean_token_accuracy": 0.7980317825011993,
+      "eval_num_tokens": 395798.0,
+      "eval_runtime": 98.6378,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 160
+    },
+    {
+      "entropy": 0.7537026956677437,
+      "epoch": 0.43425814234016885,
+      "grad_norm": 0.5521425604820251,
+      "learning_rate": 3.118091357704622e-05,
+      "loss": 0.7073616027832031,
+      "mean_token_accuracy": 0.8006261065602303,
+      "num_tokens": 446437.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43425814234016885,
+      "eval_entropy": 0.7470693678668375,
+      "eval_loss": 0.711143434047699,
+      "eval_mean_token_accuracy": 0.801459076029531,
+      "eval_num_tokens": 446437.0,
+      "eval_runtime": 98.6667,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 180
+    },
+    {
+      "entropy": 0.7328549668192863,
+      "epoch": 0.4825090470446321,
+      "grad_norm": 0.4901895821094513,
+      "learning_rate": 3.4664814535375406e-05,
+      "loss": 0.6874266624450683,
+      "mean_token_accuracy": 0.8066758885979652,
+      "num_tokens": 497682.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4825090470446321,
+      "eval_entropy": 0.7374835154983435,
+      "eval_loss": 0.6967977285385132,
+      "eval_mean_token_accuracy": 0.8055616520094068,
+      "eval_num_tokens": 497682.0,
+      "eval_runtime": 98.6234,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 200
+    },
+    {
+      "entropy": 0.7372006759047508,
+      "epoch": 0.5307599517490953,
+      "grad_norm": 0.5516473650932312,
+      "learning_rate": 3.814871549370459e-05,
+      "loss": 0.6906380653381348,
+      "mean_token_accuracy": 0.8057604253292083,
+      "num_tokens": 544670.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5307599517490953,
+      "eval_entropy": 0.710068748573239,
+      "eval_loss": 0.6860950589179993,
+      "eval_mean_token_accuracy": 0.8069355440943429,
+      "eval_num_tokens": 544670.0,
+      "eval_runtime": 98.4618,
+      "eval_samples_per_second": 14.422,
+      "eval_steps_per_second": 1.808,
+      "step": 220
+    },
+    {
+      "entropy": 0.7099859565496445,
+      "epoch": 0.5790108564535585,
+      "grad_norm": 0.457378625869751,
+      "learning_rate": 4.1632616452033785e-05,
+      "loss": 0.6674468517303467,
+      "mean_token_accuracy": 0.8087703332304954,
+      "num_tokens": 592926.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5790108564535585,
+      "eval_entropy": 0.717441116156203,
+      "eval_loss": 0.6765118837356567,
+      "eval_mean_token_accuracy": 0.8095786246690857,
+      "eval_num_tokens": 592926.0,
+      "eval_runtime": 98.6828,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 240
+    },
+    {
+      "entropy": 0.7170855522155761,
+      "epoch": 0.6272617611580217,
+      "grad_norm": 0.4604854881763458,
+      "learning_rate": 4.5116517410362974e-05,
+      "loss": 0.6647133827209473,
+      "mean_token_accuracy": 0.8115006521344185,
+      "num_tokens": 639077.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6272617611580217,
+      "eval_entropy": 0.7082702219486237,
+      "eval_loss": 0.6722468733787537,
+      "eval_mean_token_accuracy": 0.8100237367528208,
+      "eval_num_tokens": 639077.0,
+      "eval_runtime": 98.6242,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 260
+    },
+    {
+      "entropy": 0.691286925971508,
+      "epoch": 0.6755126658624849,
+      "grad_norm": 0.4626481831073761,
+      "learning_rate": 4.860041836869215e-05,
+      "loss": 0.6543298721313476,
+      "mean_token_accuracy": 0.8142751976847649,
+      "num_tokens": 691028.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6755126658624849,
+      "eval_entropy": 0.7341568747932992,
+      "eval_loss": 0.658997654914856,
+      "eval_mean_token_accuracy": 0.813439255684949,
+      "eval_num_tokens": 691028.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 280
+    },
+    {
+      "entropy": 0.691513468325138,
+      "epoch": 0.7237635705669482,
+      "grad_norm": 0.5434145331382751,
+      "learning_rate": 5.2084319327021346e-05,
+      "loss": 0.6393396377563476,
+      "mean_token_accuracy": 0.8172704800963402,
+      "num_tokens": 740255.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7237635705669482,
+      "eval_entropy": 0.699160492487168,
+      "eval_loss": 0.6522852182388306,
+      "eval_mean_token_accuracy": 0.8143363323774231,
+      "eval_num_tokens": 740255.0,
+      "eval_runtime": 98.6233,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 300
+    },
+    {
+      "entropy": 0.6918429866433143,
+      "epoch": 0.7720144752714113,
+      "grad_norm": 0.47131890058517456,
+      "learning_rate": 5.556822028535052e-05,
+      "loss": 0.6503581523895263,
+      "mean_token_accuracy": 0.8149650946259499,
+      "num_tokens": 789529.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7720144752714113,
+      "eval_entropy": 0.6949931701247611,
+      "eval_loss": 0.6464414596557617,
+      "eval_mean_token_accuracy": 0.816310279489903,
+      "eval_num_tokens": 789529.0,
+      "eval_runtime": 98.4759,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 320
+    },
+    {
+      "entropy": 0.6926439955830574,
+      "epoch": 0.8202653799758746,
+      "grad_norm": 0.46790942549705505,
+      "learning_rate": 5.905212124367972e-05,
+      "loss": 0.6453651428222656,
+      "mean_token_accuracy": 0.8171535924077034,
+      "num_tokens": 841584.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8202653799758746,
+      "eval_entropy": 0.6810062698433909,
+      "eval_loss": 0.6440736651420593,
+      "eval_mean_token_accuracy": 0.8166696479481258,
+      "eval_num_tokens": 841584.0,
+      "eval_runtime": 98.6479,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 340
+    },
+    {
+      "entropy": 0.7038845136761666,
+      "epoch": 0.8685162846803377,
+      "grad_norm": 0.5038695335388184,
+      "learning_rate": 6.253602220200891e-05,
+      "loss": 0.6558614253997803,
+      "mean_token_accuracy": 0.8127013072371483,
+      "num_tokens": 885769.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8685162846803377,
+      "eval_entropy": 0.6752923710292644,
+      "eval_loss": 0.6410524845123291,
+      "eval_mean_token_accuracy": 0.817534717950928,
+      "eval_num_tokens": 885769.0,
+      "eval_runtime": 98.5986,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 360
+    },
+    {
+      "entropy": 0.6897840186953544,
+      "epoch": 0.916767189384801,
+      "grad_norm": 0.5168097615242004,
+      "learning_rate": 6.601992316033808e-05,
+      "loss": 0.6328047752380371,
+      "mean_token_accuracy": 0.8181755512952804,
+      "num_tokens": 930056.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.916767189384801,
+      "eval_entropy": 0.6816390936964014,
+      "eval_loss": 0.6402509212493896,
+      "eval_mean_token_accuracy": 0.8172605586855599,
+      "eval_num_tokens": 930056.0,
+      "eval_runtime": 98.4787,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 380
+    },
+    {
+      "entropy": 0.6798740655183793,
+      "epoch": 0.9650180940892642,
+      "grad_norm": 0.4609147906303406,
+      "learning_rate": 6.950382411866727e-05,
+      "loss": 0.6358067989349365,
+      "mean_token_accuracy": 0.8193414479494094,
+      "num_tokens": 976892.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9650180940892642,
+      "eval_entropy": 0.6841439577970612,
+      "eval_loss": 0.6335230469703674,
+      "eval_mean_token_accuracy": 0.8190797965848045,
+      "eval_num_tokens": 976892.0,
+      "eval_runtime": 98.5983,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 400
+    },
+    {
+      "entropy": 0.6647876103719076,
+      "epoch": 1.0120627261761157,
+      "grad_norm": 0.46518462896347046,
+      "learning_rate": 7.229074030587624e-05,
+      "loss": 0.6112009525299072,
+      "mean_token_accuracy": 0.8212849283829714,
+      "num_tokens": 1024472.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0120627261761157,
+      "eval_entropy": 0.6612423688508151,
+      "eval_loss": 0.6311497688293457,
+      "eval_mean_token_accuracy": 0.8194578638237514,
+      "eval_num_tokens": 1024472.0,
+      "eval_runtime": 98.4872,
+      "eval_samples_per_second": 14.418,
+      "eval_steps_per_second": 1.807,
+      "step": 420
+    },
+    {
+      "entropy": 0.6288895070552826,
+      "epoch": 1.060313630880579,
+      "grad_norm": 0.4342021644115448,
+      "learning_rate": 7.228358026812833e-05,
+      "loss": 0.5853999614715576,
+      "mean_token_accuracy": 0.82901301831007,
+      "num_tokens": 1076964.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060313630880579,
+      "eval_entropy": 0.6567474329069759,
+      "eval_loss": 0.6245927810668945,
+      "eval_mean_token_accuracy": 0.8211770050981072,
+      "eval_num_tokens": 1076964.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 440
+    },
+    {
+      "entropy": 0.639747454226017,
+      "epoch": 1.1085645355850422,
+      "grad_norm": 0.5338298082351685,
+      "learning_rate": 7.226619357333845e-05,
+      "loss": 0.5847910404205322,
+      "mean_token_accuracy": 0.8279331669211387,
+      "num_tokens": 1126064.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1085645355850422,
+      "eval_entropy": 0.6399250211340658,
+      "eval_loss": 0.6245712637901306,
+      "eval_mean_token_accuracy": 0.8209733875949731,
+      "eval_num_tokens": 1126064.0,
+      "eval_runtime": 98.6581,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 460
+    },
+    {
+      "entropy": 0.6248803421854973,
+      "epoch": 1.1568154402895054,
+      "grad_norm": 0.4402505159378052,
+      "learning_rate": 7.223858514173552e-05,
+      "loss": 0.5847790241241455,
+      "mean_token_accuracy": 0.8273933246731758,
+      "num_tokens": 1176912.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1568154402895054,
+      "eval_entropy": 0.6459465750147787,
+      "eval_loss": 0.615716278553009,
+      "eval_mean_token_accuracy": 0.8228987813665626,
+      "eval_num_tokens": 1176912.0,
+      "eval_runtime": 98.5987,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 480
+    },
+    {
+      "entropy": 0.6356995061039925,
+      "epoch": 1.2050663449939687,
+      "grad_norm": 0.5318694114685059,
+      "learning_rate": 7.220076278617914e-05,
+      "loss": 0.5902805328369141,
+      "mean_token_accuracy": 0.8279137581586837,
+      "num_tokens": 1223383.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2050663449939687,
+      "eval_entropy": 0.6509443488013879,
+      "eval_loss": 0.6174858808517456,
+      "eval_mean_token_accuracy": 0.8229377031996009,
+      "eval_num_tokens": 1223383.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 500
+    },
+    {
+      "entropy": 0.6378119193017483,
+      "epoch": 1.2533172496984317,
+      "grad_norm": 0.5385730266571045,
+      "learning_rate": 7.215273720994853e-05,
+      "loss": 0.5850183486938476,
+      "mean_token_accuracy": 0.8282771646976471,
+      "num_tokens": 1273534.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2533172496984317,
+      "eval_entropy": 0.6438969477508845,
+      "eval_loss": 0.6134508848190308,
+      "eval_mean_token_accuracy": 0.823984161186754,
+      "eval_num_tokens": 1273534.0,
+      "eval_runtime": 98.4814,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 520
+    },
+    {
+      "entropy": 0.6330174207687378,
+      "epoch": 1.301568154402895,
+      "grad_norm": 0.4950766861438751,
+      "learning_rate": 7.209452200371367e-05,
+      "loss": 0.5886627674102783,
+      "mean_token_accuracy": 0.828557687997818,
+      "num_tokens": 1323351.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.301568154402895,
+      "eval_entropy": 0.6342737025089478,
+      "eval_loss": 0.611145555973053,
+      "eval_mean_token_accuracy": 0.8250270244780551,
+      "eval_num_tokens": 1323351.0,
+      "eval_runtime": 98.6698,
+      "eval_samples_per_second": 14.391,
+      "eval_steps_per_second": 1.804,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270076125860214,
+      "epoch": 1.3498190591073582,
+      "grad_norm": 0.435604989528656,
+      "learning_rate": 7.20261336416893e-05,
+      "loss": 0.5834732532501221,
+      "mean_token_accuracy": 0.8290600389242172,
+      "num_tokens": 1371508.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3498190591073582,
+      "eval_entropy": 0.6624904408883513,
+      "eval_loss": 0.6063718795776367,
+      "eval_mean_token_accuracy": 0.825260305002834,
+      "eval_num_tokens": 1371508.0,
+      "eval_runtime": 98.6348,
+      "eval_samples_per_second": 14.397,
+      "eval_steps_per_second": 1.805,
+      "step": 560
+    },
+    {
+      "entropy": 0.6258987173438072,
+      "epoch": 1.3980699638118215,
+      "grad_norm": 0.45945441722869873,
+      "learning_rate": 7.194759147697293e-05,
+      "loss": 0.5705549716949463,
+      "mean_token_accuracy": 0.8329991281032563,
+      "num_tokens": 1415221.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3980699638118215,
+      "eval_entropy": 0.6500632069753797,
+      "eval_loss": 0.6059838533401489,
+      "eval_mean_token_accuracy": 0.8262855604793249,
+      "eval_num_tokens": 1415221.0,
+      "eval_runtime": 98.6469,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 580
+    },
+    {
+      "entropy": 0.6301489062607288,
+      "epoch": 1.4463208685162847,
+      "grad_norm": 0.5217350721359253,
+      "learning_rate": 7.18589177360681e-05,
+      "loss": 0.5778427124023438,
+      "mean_token_accuracy": 0.8305584967136384,
+      "num_tokens": 1463356.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4463208685162847,
+      "eval_entropy": 0.6478241879618569,
+      "eval_loss": 0.6014655828475952,
+      "eval_mean_token_accuracy": 0.8265940369514937,
+      "eval_num_tokens": 1463356.0,
+      "eval_runtime": 98.6523,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 600
+    },
+    {
+      "entropy": 0.6257719069719314,
+      "epoch": 1.494571773220748,
+      "grad_norm": 0.48310142755508423,
+      "learning_rate": 7.176013751259459e-05,
+      "loss": 0.5713310241699219,
+      "mean_token_accuracy": 0.8305533796548843,
+      "num_tokens": 1512340.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.494571773220748,
+      "eval_entropy": 0.6396401956510008,
+      "eval_loss": 0.5978795289993286,
+      "eval_mean_token_accuracy": 0.8273170630583603,
+      "eval_num_tokens": 1512340.0,
+      "eval_runtime": 98.4958,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 620
+    },
+    {
+      "entropy": 0.6162159442901611,
+      "epoch": 1.5428226779252112,
+      "grad_norm": 0.4500057101249695,
+      "learning_rate": 7.165127876018717e-05,
+      "loss": 0.565293264389038,
+      "mean_token_accuracy": 0.8311659932136536,
+      "num_tokens": 1560911.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5428226779252112,
+      "eval_entropy": 0.6381519051750054,
+      "eval_loss": 0.5963984131813049,
+      "eval_mean_token_accuracy": 0.8268409247478742,
+      "eval_num_tokens": 1560911.0,
+      "eval_runtime": 98.6803,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 640
+    },
+    {
+      "entropy": 0.6088747605681419,
+      "epoch": 1.5910735826296745,
+      "grad_norm": 0.46954667568206787,
+      "learning_rate": 7.153237228458511e-05,
+      "loss": 0.5602522850036621,
+      "mean_token_accuracy": 0.8345131903886795,
+      "num_tokens": 1609409.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5910735826296745,
+      "eval_entropy": 0.6413098875056492,
+      "eval_loss": 0.5938988327980042,
+      "eval_mean_token_accuracy": 0.828771137119679,
+      "eval_num_tokens": 1609409.0,
+      "eval_runtime": 98.4951,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 660
+    },
+    {
+      "entropy": 0.6209868296980858,
+      "epoch": 1.6393244873341375,
+      "grad_norm": 0.3867878317832947,
+      "learning_rate": 7.140345173491446e-05,
+      "loss": 0.5663797378540039,
+      "mean_token_accuracy": 0.8352224260568619,
+      "num_tokens": 1657430.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.6393244873341375,
+      "eval_entropy": 0.6182791748073664,
+      "eval_loss": 0.5919068455696106,
+      "eval_mean_token_accuracy": 0.8286421476455217,
+      "eval_num_tokens": 1657430.0,
+      "eval_runtime": 98.6211,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 680
+    },
+    {
+      "entropy": 0.6191923573613167,
+      "epoch": 1.6875753920386007,
+      "grad_norm": 0.40004226565361023,
+      "learning_rate": 7.12645535941658e-05,
+      "loss": 0.57517991065979,
+      "mean_token_accuracy": 0.8333286643028259,
+      "num_tokens": 1708058.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.6875753920386007,
+      "eval_entropy": 0.6244879199212856,
+      "eval_loss": 0.5897895693778992,
+      "eval_mean_token_accuracy": 0.8293621252092083,
+      "eval_num_tokens": 1708058.0,
+      "eval_runtime": 98.4974,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 700
+    },
+    {
+      "entropy": 0.6199282392859459,
+      "epoch": 1.7358262967430638,
+      "grad_norm": 0.36575791239738464,
+      "learning_rate": 7.111571716886997e-05,
+      "loss": 0.5717356681823731,
+      "mean_token_accuracy": 0.8321028783917427,
+      "num_tokens": 1761262.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.7358262967430638,
+      "eval_entropy": 0.6162440127201294,
+      "eval_loss": 0.5864423513412476,
+      "eval_mean_token_accuracy": 0.830874237786518,
+      "eval_num_tokens": 1761262.0,
+      "eval_runtime": 98.6109,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 720
+    },
+    {
+      "entropy": 0.6127455830574036,
+      "epoch": 1.784077201447527,
+      "grad_norm": 0.4778473973274231,
+      "learning_rate": 7.095698457797475e-05,
+      "loss": 0.5664590835571289,
+      "mean_token_accuracy": 0.8324130088090896,
+      "num_tokens": 1812127.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.784077201447527,
+      "eval_entropy": 0.6273092945640006,
+      "eval_loss": 0.5847376585006714,
+      "eval_mean_token_accuracy": 0.831051754817534,
+      "eval_num_tokens": 1812127.0,
+      "eval_runtime": 98.6513,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 740
+    },
+    {
+      "entropy": 0.6141144648194313,
+      "epoch": 1.8323281061519903,
+      "grad_norm": 0.6467089056968689,
+      "learning_rate": 7.078840074092574e-05,
+      "loss": 0.5548617362976074,
+      "mean_token_accuracy": 0.8342376738786698,
+      "num_tokens": 1858403.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.8323281061519903,
+      "eval_entropy": 0.5951976082967908,
+      "eval_loss": 0.5850171446800232,
+      "eval_mean_token_accuracy": 0.8303041870004675,
+      "eval_num_tokens": 1858403.0,
+      "eval_runtime": 98.6601,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 760
+    },
+    {
+      "entropy": 0.5961867064237595,
+      "epoch": 1.8805790108564535,
+      "grad_norm": 0.41280242800712585,
+      "learning_rate": 7.061001336495465e-05,
+      "loss": 0.5511267185211182,
+      "mean_token_accuracy": 0.8392465204000473,
+      "num_tokens": 1907361.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8805790108564535,
+      "eval_entropy": 0.6167017998320333,
+      "eval_loss": 0.5826791524887085,
+      "eval_mean_token_accuracy": 0.8309678050239434,
+      "eval_num_tokens": 1907361.0,
+      "eval_runtime": 98.6564,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 780
+    },
+    {
+      "entropy": 0.6025628983974457,
+      "epoch": 1.9288299155609168,
+      "grad_norm": 0.4078947901725769,
+      "learning_rate": 7.042187293157881e-05,
+      "loss": 0.5552026271820069,
+      "mean_token_accuracy": 0.8359778299927711,
+      "num_tokens": 1956605.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9288299155609168,
+      "eval_entropy": 0.6047008484601974,
+      "eval_loss": 0.5811123847961426,
+      "eval_mean_token_accuracy": 0.8320730804057603,
+      "eval_num_tokens": 1956605.0,
+      "eval_runtime": 98.6033,
+      "eval_samples_per_second": 14.401,
+      "eval_steps_per_second": 1.805,
+      "step": 800
+    },
+    {
+      "entropy": 0.6041995719075203,
+      "epoch": 1.97708082026538,
+      "grad_norm": 0.543890118598938,
+      "learning_rate": 7.022403268231535e-05,
+      "loss": 0.554384422302246,
+      "mean_token_accuracy": 0.8380729153752327,
+      "num_tokens": 2003308.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.97708082026538,
+      "eval_entropy": 0.6005127917179901,
+      "eval_loss": 0.5800131559371948,
+      "eval_mean_token_accuracy": 0.8313186547059691,
+      "eval_num_tokens": 2003308.0,
+      "eval_runtime": 98.4119,
+      "eval_samples_per_second": 14.429,
+      "eval_steps_per_second": 1.809,
+      "step": 820
+    },
+    {
+      "entropy": 0.567442889396961,
+      "epoch": 2.0241254523522314,
+      "grad_norm": 0.5015465021133423,
+      "learning_rate": 7.001654860361456e-05,
+      "loss": 0.5208251476287842,
+      "mean_token_accuracy": 0.8442784960453327,
+      "num_tokens": 2051331.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0241254523522314,
+      "eval_entropy": 0.5785037809878253,
+      "eval_loss": 0.5865591168403625,
+      "eval_mean_token_accuracy": 0.8307993857378371,
+      "eval_num_tokens": 2051331.0,
+      "eval_runtime": 98.5061,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 840
+    },
+    {
+      "entropy": 0.5552012033760547,
+      "epoch": 2.0723763570566947,
+      "grad_norm": 0.5975943207740784,
+      "learning_rate": 6.979947941101646e-05,
+      "loss": 0.5064489841461182,
+      "mean_token_accuracy": 0.8468396887183189,
+      "num_tokens": 2101528.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0723763570566947,
+      "eval_entropy": 0.5844205239515626,
+      "eval_loss": 0.585284948348999,
+      "eval_mean_token_accuracy": 0.8309797065311604,
+      "eval_num_tokens": 2101528.0,
+      "eval_runtime": 98.4982,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 860
+    },
+    {
+      "entropy": 0.5401638634502888,
+      "epoch": 2.120627261761158,
+      "grad_norm": 0.4194582998752594,
+      "learning_rate": 6.957288653253486e-05,
+      "loss": 0.49037814140319824,
+      "mean_token_accuracy": 0.8528496414422989,
+      "num_tokens": 2149550.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.120627261761158,
+      "eval_entropy": 0.574507936835289,
+      "eval_loss": 0.5836747288703918,
+      "eval_mean_token_accuracy": 0.8325241643391298,
+      "eval_num_tokens": 2149550.0,
+      "eval_runtime": 98.5592,
+      "eval_samples_per_second": 14.408,
+      "eval_steps_per_second": 1.806,
+      "step": 880
+    },
+    {
+      "entropy": 0.5606910541653634,
+      "epoch": 2.168878166465621,
+      "grad_norm": 0.5395355820655823,
+      "learning_rate": 6.933683409127401e-05,
+      "loss": 0.5089571475982666,
+      "mean_token_accuracy": 0.8459660410881042,
+      "num_tokens": 2202162.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.168878166465621,
+      "eval_entropy": 0.5779740023479033,
+      "eval_loss": 0.5816043615341187,
+      "eval_mean_token_accuracy": 0.8326318036304431,
+      "eval_num_tokens": 2202162.0,
+      "eval_runtime": 98.4386,
+      "eval_samples_per_second": 14.425,
+      "eval_steps_per_second": 1.808,
+      "step": 900
+    },
+    {
+      "entropy": 0.5380903735756875,
+      "epoch": 2.2171290711700844,
+      "grad_norm": 0.46588417887687683,
+      "learning_rate": 6.909138888728257e-05,
+      "loss": 0.4902662754058838,
+      "mean_token_accuracy": 0.8493925213813782,
+      "num_tokens": 2256539.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2171290711700844,
+      "eval_entropy": 0.570154261220707,
+      "eval_loss": 0.5827520489692688,
+      "eval_mean_token_accuracy": 0.8321914167216654,
+      "eval_num_tokens": 2256539.0,
+      "eval_runtime": 98.5674,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 920
+    },
+    {
+      "entropy": 0.558555257320404,
+      "epoch": 2.2653799758745476,
+      "grad_norm": 0.44576048851013184,
+      "learning_rate": 6.883662037864996e-05,
+      "loss": 0.5000699520111084,
+      "mean_token_accuracy": 0.8475695163011551,
+      "num_tokens": 2299698.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2653799758745476,
+      "eval_entropy": 0.5715417749761196,
+      "eval_loss": 0.5831062197685242,
+      "eval_mean_token_accuracy": 0.8327409899636601,
+      "eval_num_tokens": 2299698.0,
+      "eval_runtime": 98.568,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 940
+    },
+    {
+      "entropy": 0.5582522705197335,
+      "epoch": 2.313630880579011,
+      "grad_norm": 0.38966405391693115,
+      "learning_rate": 6.857260066185045e-05,
+      "loss": 0.5132341861724854,
+      "mean_token_accuracy": 0.8466864034533501,
+      "num_tokens": 2347134.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.313630880579011,
+      "eval_entropy": 0.5823918579334624,
+      "eval_loss": 0.5797588229179382,
+      "eval_mean_token_accuracy": 0.8326037566313583,
+      "eval_num_tokens": 2347134.0,
+      "eval_runtime": 98.6222,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 960
+    },
+    {
+      "entropy": 0.5557353265583516,
+      "epoch": 2.361881785283474,
+      "grad_norm": 0.42300811409950256,
+      "learning_rate": 6.829940445134089e-05,
+      "loss": 0.5035375118255615,
+      "mean_token_accuracy": 0.8479988545179367,
+      "num_tokens": 2394568.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.361881785283474,
+      "eval_entropy": 0.5542296482605881,
+      "eval_loss": 0.5800604820251465,
+      "eval_mean_token_accuracy": 0.8336404163515969,
+      "eval_num_tokens": 2394568.0,
+      "eval_runtime": 98.615,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 980
+    },
+    {
+      "entropy": 0.5492161035537719,
+      "epoch": 2.4101326899879374,
+      "grad_norm": 0.47482752799987793,
+      "learning_rate": 6.801710905841724e-05,
+      "loss": 0.5032052040100098,
+      "mean_token_accuracy": 0.8488596931099892,
+      "num_tokens": 2443253.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4101326899879374,
+      "eval_entropy": 0.5755258423558781,
+      "eval_loss": 0.5764774680137634,
+      "eval_mean_token_accuracy": 0.8336571763070781,
+      "eval_num_tokens": 2443253.0,
+      "eval_runtime": 98.4765,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5439704649150372,
+      "epoch": 2.4583835946924006,
+      "grad_norm": 0.3656543493270874,
+      "learning_rate": 6.772579436933644e-05,
+      "loss": 0.49903340339660646,
+      "mean_token_accuracy": 0.8499931409955025,
+      "num_tokens": 2495178.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4583835946924006,
+      "eval_entropy": 0.5712662409530597,
+      "eval_loss": 0.5777108073234558,
+      "eval_mean_token_accuracy": 0.8336941352051296,
+      "eval_num_tokens": 2495178.0,
+      "eval_runtime": 98.678,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 1020
+    },
+    {
+      "entropy": 0.5448883637785912,
+      "epoch": 2.5066344993968634,
+      "grad_norm": 0.3990268111228943,
+      "learning_rate": 6.742554282270959e-05,
+      "loss": 0.4990490436553955,
+      "mean_token_accuracy": 0.8488436549901962,
+      "num_tokens": 2541733.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5066344993968634,
+      "eval_entropy": 0.5816385483139017,
+      "eval_loss": 0.5749375820159912,
+      "eval_mean_token_accuracy": 0.833801462074344,
+      "eval_num_tokens": 2541733.0,
+      "eval_runtime": 98.6209,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 1040
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.27477297872896e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1060/trainer_state.json

@@ -0,0 +1,1147 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.554885404101327,
+  "eval_steps": 20,
+  "global_step": 1060,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    },
+    {
+      "entropy": 0.8518766984343529,
+      "epoch": 0.28950542822677927,
+      "grad_norm": 0.635079026222229,
+      "learning_rate": 2.0729210702058663e-05,
+      "loss": 0.8058656692504883,
+      "mean_token_accuracy": 0.7847469851374627,
+      "num_tokens": 298041.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28950542822677927,
+      "eval_entropy": 0.838603059227547,
+      "eval_loss": 0.7992516756057739,
+      "eval_mean_token_accuracy": 0.7849764954508021,
+      "eval_num_tokens": 298041.0,
+      "eval_runtime": 98.5627,
+      "eval_samples_per_second": 14.407,
+      "eval_steps_per_second": 1.806,
+      "step": 120
+    },
+    {
+      "entropy": 0.8069365546107292,
+      "epoch": 0.33775633293124246,
+      "grad_norm": 0.5280472636222839,
+      "learning_rate": 2.421311166038785e-05,
+      "loss": 0.7562496662139893,
+      "mean_token_accuracy": 0.7897997036576271,
+      "num_tokens": 348955.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33775633293124246,
+      "eval_entropy": 0.8161682168419442,
+      "eval_loss": 0.7599642276763916,
+      "eval_mean_token_accuracy": 0.7912511644738444,
+      "eval_num_tokens": 348955.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 140
+    },
+    {
+      "entropy": 0.7952415809035301,
+      "epoch": 0.38600723763570566,
+      "grad_norm": 0.5387083888053894,
+      "learning_rate": 2.7697012618717034e-05,
+      "loss": 0.740808629989624,
+      "mean_token_accuracy": 0.7941806614398956,
+      "num_tokens": 395798.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38600723763570566,
+      "eval_entropy": 0.7856516191798649,
+      "eval_loss": 0.7324908971786499,
+      "eval_mean_token_accuracy": 0.7980317825011993,
+      "eval_num_tokens": 395798.0,
+      "eval_runtime": 98.6378,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 160
+    },
+    {
+      "entropy": 0.7537026956677437,
+      "epoch": 0.43425814234016885,
+      "grad_norm": 0.5521425604820251,
+      "learning_rate": 3.118091357704622e-05,
+      "loss": 0.7073616027832031,
+      "mean_token_accuracy": 0.8006261065602303,
+      "num_tokens": 446437.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43425814234016885,
+      "eval_entropy": 0.7470693678668375,
+      "eval_loss": 0.711143434047699,
+      "eval_mean_token_accuracy": 0.801459076029531,
+      "eval_num_tokens": 446437.0,
+      "eval_runtime": 98.6667,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 180
+    },
+    {
+      "entropy": 0.7328549668192863,
+      "epoch": 0.4825090470446321,
+      "grad_norm": 0.4901895821094513,
+      "learning_rate": 3.4664814535375406e-05,
+      "loss": 0.6874266624450683,
+      "mean_token_accuracy": 0.8066758885979652,
+      "num_tokens": 497682.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4825090470446321,
+      "eval_entropy": 0.7374835154983435,
+      "eval_loss": 0.6967977285385132,
+      "eval_mean_token_accuracy": 0.8055616520094068,
+      "eval_num_tokens": 497682.0,
+      "eval_runtime": 98.6234,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 200
+    },
+    {
+      "entropy": 0.7372006759047508,
+      "epoch": 0.5307599517490953,
+      "grad_norm": 0.5516473650932312,
+      "learning_rate": 3.814871549370459e-05,
+      "loss": 0.6906380653381348,
+      "mean_token_accuracy": 0.8057604253292083,
+      "num_tokens": 544670.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5307599517490953,
+      "eval_entropy": 0.710068748573239,
+      "eval_loss": 0.6860950589179993,
+      "eval_mean_token_accuracy": 0.8069355440943429,
+      "eval_num_tokens": 544670.0,
+      "eval_runtime": 98.4618,
+      "eval_samples_per_second": 14.422,
+      "eval_steps_per_second": 1.808,
+      "step": 220
+    },
+    {
+      "entropy": 0.7099859565496445,
+      "epoch": 0.5790108564535585,
+      "grad_norm": 0.457378625869751,
+      "learning_rate": 4.1632616452033785e-05,
+      "loss": 0.6674468517303467,
+      "mean_token_accuracy": 0.8087703332304954,
+      "num_tokens": 592926.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5790108564535585,
+      "eval_entropy": 0.717441116156203,
+      "eval_loss": 0.6765118837356567,
+      "eval_mean_token_accuracy": 0.8095786246690857,
+      "eval_num_tokens": 592926.0,
+      "eval_runtime": 98.6828,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 240
+    },
+    {
+      "entropy": 0.7170855522155761,
+      "epoch": 0.6272617611580217,
+      "grad_norm": 0.4604854881763458,
+      "learning_rate": 4.5116517410362974e-05,
+      "loss": 0.6647133827209473,
+      "mean_token_accuracy": 0.8115006521344185,
+      "num_tokens": 639077.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6272617611580217,
+      "eval_entropy": 0.7082702219486237,
+      "eval_loss": 0.6722468733787537,
+      "eval_mean_token_accuracy": 0.8100237367528208,
+      "eval_num_tokens": 639077.0,
+      "eval_runtime": 98.6242,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 260
+    },
+    {
+      "entropy": 0.691286925971508,
+      "epoch": 0.6755126658624849,
+      "grad_norm": 0.4626481831073761,
+      "learning_rate": 4.860041836869215e-05,
+      "loss": 0.6543298721313476,
+      "mean_token_accuracy": 0.8142751976847649,
+      "num_tokens": 691028.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6755126658624849,
+      "eval_entropy": 0.7341568747932992,
+      "eval_loss": 0.658997654914856,
+      "eval_mean_token_accuracy": 0.813439255684949,
+      "eval_num_tokens": 691028.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 280
+    },
+    {
+      "entropy": 0.691513468325138,
+      "epoch": 0.7237635705669482,
+      "grad_norm": 0.5434145331382751,
+      "learning_rate": 5.2084319327021346e-05,
+      "loss": 0.6393396377563476,
+      "mean_token_accuracy": 0.8172704800963402,
+      "num_tokens": 740255.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7237635705669482,
+      "eval_entropy": 0.699160492487168,
+      "eval_loss": 0.6522852182388306,
+      "eval_mean_token_accuracy": 0.8143363323774231,
+      "eval_num_tokens": 740255.0,
+      "eval_runtime": 98.6233,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 300
+    },
+    {
+      "entropy": 0.6918429866433143,
+      "epoch": 0.7720144752714113,
+      "grad_norm": 0.47131890058517456,
+      "learning_rate": 5.556822028535052e-05,
+      "loss": 0.6503581523895263,
+      "mean_token_accuracy": 0.8149650946259499,
+      "num_tokens": 789529.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7720144752714113,
+      "eval_entropy": 0.6949931701247611,
+      "eval_loss": 0.6464414596557617,
+      "eval_mean_token_accuracy": 0.816310279489903,
+      "eval_num_tokens": 789529.0,
+      "eval_runtime": 98.4759,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 320
+    },
+    {
+      "entropy": 0.6926439955830574,
+      "epoch": 0.8202653799758746,
+      "grad_norm": 0.46790942549705505,
+      "learning_rate": 5.905212124367972e-05,
+      "loss": 0.6453651428222656,
+      "mean_token_accuracy": 0.8171535924077034,
+      "num_tokens": 841584.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8202653799758746,
+      "eval_entropy": 0.6810062698433909,
+      "eval_loss": 0.6440736651420593,
+      "eval_mean_token_accuracy": 0.8166696479481258,
+      "eval_num_tokens": 841584.0,
+      "eval_runtime": 98.6479,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 340
+    },
+    {
+      "entropy": 0.7038845136761666,
+      "epoch": 0.8685162846803377,
+      "grad_norm": 0.5038695335388184,
+      "learning_rate": 6.253602220200891e-05,
+      "loss": 0.6558614253997803,
+      "mean_token_accuracy": 0.8127013072371483,
+      "num_tokens": 885769.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8685162846803377,
+      "eval_entropy": 0.6752923710292644,
+      "eval_loss": 0.6410524845123291,
+      "eval_mean_token_accuracy": 0.817534717950928,
+      "eval_num_tokens": 885769.0,
+      "eval_runtime": 98.5986,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 360
+    },
+    {
+      "entropy": 0.6897840186953544,
+      "epoch": 0.916767189384801,
+      "grad_norm": 0.5168097615242004,
+      "learning_rate": 6.601992316033808e-05,
+      "loss": 0.6328047752380371,
+      "mean_token_accuracy": 0.8181755512952804,
+      "num_tokens": 930056.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.916767189384801,
+      "eval_entropy": 0.6816390936964014,
+      "eval_loss": 0.6402509212493896,
+      "eval_mean_token_accuracy": 0.8172605586855599,
+      "eval_num_tokens": 930056.0,
+      "eval_runtime": 98.4787,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 380
+    },
+    {
+      "entropy": 0.6798740655183793,
+      "epoch": 0.9650180940892642,
+      "grad_norm": 0.4609147906303406,
+      "learning_rate": 6.950382411866727e-05,
+      "loss": 0.6358067989349365,
+      "mean_token_accuracy": 0.8193414479494094,
+      "num_tokens": 976892.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9650180940892642,
+      "eval_entropy": 0.6841439577970612,
+      "eval_loss": 0.6335230469703674,
+      "eval_mean_token_accuracy": 0.8190797965848045,
+      "eval_num_tokens": 976892.0,
+      "eval_runtime": 98.5983,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 400
+    },
+    {
+      "entropy": 0.6647876103719076,
+      "epoch": 1.0120627261761157,
+      "grad_norm": 0.46518462896347046,
+      "learning_rate": 7.229074030587624e-05,
+      "loss": 0.6112009525299072,
+      "mean_token_accuracy": 0.8212849283829714,
+      "num_tokens": 1024472.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0120627261761157,
+      "eval_entropy": 0.6612423688508151,
+      "eval_loss": 0.6311497688293457,
+      "eval_mean_token_accuracy": 0.8194578638237514,
+      "eval_num_tokens": 1024472.0,
+      "eval_runtime": 98.4872,
+      "eval_samples_per_second": 14.418,
+      "eval_steps_per_second": 1.807,
+      "step": 420
+    },
+    {
+      "entropy": 0.6288895070552826,
+      "epoch": 1.060313630880579,
+      "grad_norm": 0.4342021644115448,
+      "learning_rate": 7.228358026812833e-05,
+      "loss": 0.5853999614715576,
+      "mean_token_accuracy": 0.82901301831007,
+      "num_tokens": 1076964.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060313630880579,
+      "eval_entropy": 0.6567474329069759,
+      "eval_loss": 0.6245927810668945,
+      "eval_mean_token_accuracy": 0.8211770050981072,
+      "eval_num_tokens": 1076964.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 440
+    },
+    {
+      "entropy": 0.639747454226017,
+      "epoch": 1.1085645355850422,
+      "grad_norm": 0.5338298082351685,
+      "learning_rate": 7.226619357333845e-05,
+      "loss": 0.5847910404205322,
+      "mean_token_accuracy": 0.8279331669211387,
+      "num_tokens": 1126064.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1085645355850422,
+      "eval_entropy": 0.6399250211340658,
+      "eval_loss": 0.6245712637901306,
+      "eval_mean_token_accuracy": 0.8209733875949731,
+      "eval_num_tokens": 1126064.0,
+      "eval_runtime": 98.6581,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 460
+    },
+    {
+      "entropy": 0.6248803421854973,
+      "epoch": 1.1568154402895054,
+      "grad_norm": 0.4402505159378052,
+      "learning_rate": 7.223858514173552e-05,
+      "loss": 0.5847790241241455,
+      "mean_token_accuracy": 0.8273933246731758,
+      "num_tokens": 1176912.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1568154402895054,
+      "eval_entropy": 0.6459465750147787,
+      "eval_loss": 0.615716278553009,
+      "eval_mean_token_accuracy": 0.8228987813665626,
+      "eval_num_tokens": 1176912.0,
+      "eval_runtime": 98.5987,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 480
+    },
+    {
+      "entropy": 0.6356995061039925,
+      "epoch": 1.2050663449939687,
+      "grad_norm": 0.5318694114685059,
+      "learning_rate": 7.220076278617914e-05,
+      "loss": 0.5902805328369141,
+      "mean_token_accuracy": 0.8279137581586837,
+      "num_tokens": 1223383.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2050663449939687,
+      "eval_entropy": 0.6509443488013879,
+      "eval_loss": 0.6174858808517456,
+      "eval_mean_token_accuracy": 0.8229377031996009,
+      "eval_num_tokens": 1223383.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 500
+    },
+    {
+      "entropy": 0.6378119193017483,
+      "epoch": 1.2533172496984317,
+      "grad_norm": 0.5385730266571045,
+      "learning_rate": 7.215273720994853e-05,
+      "loss": 0.5850183486938476,
+      "mean_token_accuracy": 0.8282771646976471,
+      "num_tokens": 1273534.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2533172496984317,
+      "eval_entropy": 0.6438969477508845,
+      "eval_loss": 0.6134508848190308,
+      "eval_mean_token_accuracy": 0.823984161186754,
+      "eval_num_tokens": 1273534.0,
+      "eval_runtime": 98.4814,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 520
+    },
+    {
+      "entropy": 0.6330174207687378,
+      "epoch": 1.301568154402895,
+      "grad_norm": 0.4950766861438751,
+      "learning_rate": 7.209452200371367e-05,
+      "loss": 0.5886627674102783,
+      "mean_token_accuracy": 0.828557687997818,
+      "num_tokens": 1323351.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.301568154402895,
+      "eval_entropy": 0.6342737025089478,
+      "eval_loss": 0.611145555973053,
+      "eval_mean_token_accuracy": 0.8250270244780551,
+      "eval_num_tokens": 1323351.0,
+      "eval_runtime": 98.6698,
+      "eval_samples_per_second": 14.391,
+      "eval_steps_per_second": 1.804,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270076125860214,
+      "epoch": 1.3498190591073582,
+      "grad_norm": 0.435604989528656,
+      "learning_rate": 7.20261336416893e-05,
+      "loss": 0.5834732532501221,
+      "mean_token_accuracy": 0.8290600389242172,
+      "num_tokens": 1371508.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3498190591073582,
+      "eval_entropy": 0.6624904408883513,
+      "eval_loss": 0.6063718795776367,
+      "eval_mean_token_accuracy": 0.825260305002834,
+      "eval_num_tokens": 1371508.0,
+      "eval_runtime": 98.6348,
+      "eval_samples_per_second": 14.397,
+      "eval_steps_per_second": 1.805,
+      "step": 560
+    },
+    {
+      "entropy": 0.6258987173438072,
+      "epoch": 1.3980699638118215,
+      "grad_norm": 0.45945441722869873,
+      "learning_rate": 7.194759147697293e-05,
+      "loss": 0.5705549716949463,
+      "mean_token_accuracy": 0.8329991281032563,
+      "num_tokens": 1415221.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3980699638118215,
+      "eval_entropy": 0.6500632069753797,
+      "eval_loss": 0.6059838533401489,
+      "eval_mean_token_accuracy": 0.8262855604793249,
+      "eval_num_tokens": 1415221.0,
+      "eval_runtime": 98.6469,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 580
+    },
+    {
+      "entropy": 0.6301489062607288,
+      "epoch": 1.4463208685162847,
+      "grad_norm": 0.5217350721359253,
+      "learning_rate": 7.18589177360681e-05,
+      "loss": 0.5778427124023438,
+      "mean_token_accuracy": 0.8305584967136384,
+      "num_tokens": 1463356.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4463208685162847,
+      "eval_entropy": 0.6478241879618569,
+      "eval_loss": 0.6014655828475952,
+      "eval_mean_token_accuracy": 0.8265940369514937,
+      "eval_num_tokens": 1463356.0,
+      "eval_runtime": 98.6523,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 600
+    },
+    {
+      "entropy": 0.6257719069719314,
+      "epoch": 1.494571773220748,
+      "grad_norm": 0.48310142755508423,
+      "learning_rate": 7.176013751259459e-05,
+      "loss": 0.5713310241699219,
+      "mean_token_accuracy": 0.8305533796548843,
+      "num_tokens": 1512340.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.494571773220748,
+      "eval_entropy": 0.6396401956510008,
+      "eval_loss": 0.5978795289993286,
+      "eval_mean_token_accuracy": 0.8273170630583603,
+      "eval_num_tokens": 1512340.0,
+      "eval_runtime": 98.4958,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 620
+    },
+    {
+      "entropy": 0.6162159442901611,
+      "epoch": 1.5428226779252112,
+      "grad_norm": 0.4500057101249695,
+      "learning_rate": 7.165127876018717e-05,
+      "loss": 0.565293264389038,
+      "mean_token_accuracy": 0.8311659932136536,
+      "num_tokens": 1560911.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5428226779252112,
+      "eval_entropy": 0.6381519051750054,
+      "eval_loss": 0.5963984131813049,
+      "eval_mean_token_accuracy": 0.8268409247478742,
+      "eval_num_tokens": 1560911.0,
+      "eval_runtime": 98.6803,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 640
+    },
+    {
+      "entropy": 0.6088747605681419,
+      "epoch": 1.5910735826296745,
+      "grad_norm": 0.46954667568206787,
+      "learning_rate": 7.153237228458511e-05,
+      "loss": 0.5602522850036621,
+      "mean_token_accuracy": 0.8345131903886795,
+      "num_tokens": 1609409.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5910735826296745,
+      "eval_entropy": 0.6413098875056492,
+      "eval_loss": 0.5938988327980042,
+      "eval_mean_token_accuracy": 0.828771137119679,
+      "eval_num_tokens": 1609409.0,
+      "eval_runtime": 98.4951,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 660
+    },
+    {
+      "entropy": 0.6209868296980858,
+      "epoch": 1.6393244873341375,
+      "grad_norm": 0.3867878317832947,
+      "learning_rate": 7.140345173491446e-05,
+      "loss": 0.5663797378540039,
+      "mean_token_accuracy": 0.8352224260568619,
+      "num_tokens": 1657430.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.6393244873341375,
+      "eval_entropy": 0.6182791748073664,
+      "eval_loss": 0.5919068455696106,
+      "eval_mean_token_accuracy": 0.8286421476455217,
+      "eval_num_tokens": 1657430.0,
+      "eval_runtime": 98.6211,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 680
+    },
+    {
+      "entropy": 0.6191923573613167,
+      "epoch": 1.6875753920386007,
+      "grad_norm": 0.40004226565361023,
+      "learning_rate": 7.12645535941658e-05,
+      "loss": 0.57517991065979,
+      "mean_token_accuracy": 0.8333286643028259,
+      "num_tokens": 1708058.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.6875753920386007,
+      "eval_entropy": 0.6244879199212856,
+      "eval_loss": 0.5897895693778992,
+      "eval_mean_token_accuracy": 0.8293621252092083,
+      "eval_num_tokens": 1708058.0,
+      "eval_runtime": 98.4974,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 700
+    },
+    {
+      "entropy": 0.6199282392859459,
+      "epoch": 1.7358262967430638,
+      "grad_norm": 0.36575791239738464,
+      "learning_rate": 7.111571716886997e-05,
+      "loss": 0.5717356681823731,
+      "mean_token_accuracy": 0.8321028783917427,
+      "num_tokens": 1761262.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.7358262967430638,
+      "eval_entropy": 0.6162440127201294,
+      "eval_loss": 0.5864423513412476,
+      "eval_mean_token_accuracy": 0.830874237786518,
+      "eval_num_tokens": 1761262.0,
+      "eval_runtime": 98.6109,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 720
+    },
+    {
+      "entropy": 0.6127455830574036,
+      "epoch": 1.784077201447527,
+      "grad_norm": 0.4778473973274231,
+      "learning_rate": 7.095698457797475e-05,
+      "loss": 0.5664590835571289,
+      "mean_token_accuracy": 0.8324130088090896,
+      "num_tokens": 1812127.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.784077201447527,
+      "eval_entropy": 0.6273092945640006,
+      "eval_loss": 0.5847376585006714,
+      "eval_mean_token_accuracy": 0.831051754817534,
+      "eval_num_tokens": 1812127.0,
+      "eval_runtime": 98.6513,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 740
+    },
+    {
+      "entropy": 0.6141144648194313,
+      "epoch": 1.8323281061519903,
+      "grad_norm": 0.6467089056968689,
+      "learning_rate": 7.078840074092574e-05,
+      "loss": 0.5548617362976074,
+      "mean_token_accuracy": 0.8342376738786698,
+      "num_tokens": 1858403.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.8323281061519903,
+      "eval_entropy": 0.5951976082967908,
+      "eval_loss": 0.5850171446800232,
+      "eval_mean_token_accuracy": 0.8303041870004675,
+      "eval_num_tokens": 1858403.0,
+      "eval_runtime": 98.6601,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 760
+    },
+    {
+      "entropy": 0.5961867064237595,
+      "epoch": 1.8805790108564535,
+      "grad_norm": 0.41280242800712585,
+      "learning_rate": 7.061001336495465e-05,
+      "loss": 0.5511267185211182,
+      "mean_token_accuracy": 0.8392465204000473,
+      "num_tokens": 1907361.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8805790108564535,
+      "eval_entropy": 0.6167017998320333,
+      "eval_loss": 0.5826791524887085,
+      "eval_mean_token_accuracy": 0.8309678050239434,
+      "eval_num_tokens": 1907361.0,
+      "eval_runtime": 98.6564,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 780
+    },
+    {
+      "entropy": 0.6025628983974457,
+      "epoch": 1.9288299155609168,
+      "grad_norm": 0.4078947901725769,
+      "learning_rate": 7.042187293157881e-05,
+      "loss": 0.5552026271820069,
+      "mean_token_accuracy": 0.8359778299927711,
+      "num_tokens": 1956605.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9288299155609168,
+      "eval_entropy": 0.6047008484601974,
+      "eval_loss": 0.5811123847961426,
+      "eval_mean_token_accuracy": 0.8320730804057603,
+      "eval_num_tokens": 1956605.0,
+      "eval_runtime": 98.6033,
+      "eval_samples_per_second": 14.401,
+      "eval_steps_per_second": 1.805,
+      "step": 800
+    },
+    {
+      "entropy": 0.6041995719075203,
+      "epoch": 1.97708082026538,
+      "grad_norm": 0.543890118598938,
+      "learning_rate": 7.022403268231535e-05,
+      "loss": 0.554384422302246,
+      "mean_token_accuracy": 0.8380729153752327,
+      "num_tokens": 2003308.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.97708082026538,
+      "eval_entropy": 0.6005127917179901,
+      "eval_loss": 0.5800131559371948,
+      "eval_mean_token_accuracy": 0.8313186547059691,
+      "eval_num_tokens": 2003308.0,
+      "eval_runtime": 98.4119,
+      "eval_samples_per_second": 14.429,
+      "eval_steps_per_second": 1.809,
+      "step": 820
+    },
+    {
+      "entropy": 0.567442889396961,
+      "epoch": 2.0241254523522314,
+      "grad_norm": 0.5015465021133423,
+      "learning_rate": 7.001654860361456e-05,
+      "loss": 0.5208251476287842,
+      "mean_token_accuracy": 0.8442784960453327,
+      "num_tokens": 2051331.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0241254523522314,
+      "eval_entropy": 0.5785037809878253,
+      "eval_loss": 0.5865591168403625,
+      "eval_mean_token_accuracy": 0.8307993857378371,
+      "eval_num_tokens": 2051331.0,
+      "eval_runtime": 98.5061,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 840
+    },
+    {
+      "entropy": 0.5552012033760547,
+      "epoch": 2.0723763570566947,
+      "grad_norm": 0.5975943207740784,
+      "learning_rate": 6.979947941101646e-05,
+      "loss": 0.5064489841461182,
+      "mean_token_accuracy": 0.8468396887183189,
+      "num_tokens": 2101528.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0723763570566947,
+      "eval_entropy": 0.5844205239515626,
+      "eval_loss": 0.585284948348999,
+      "eval_mean_token_accuracy": 0.8309797065311604,
+      "eval_num_tokens": 2101528.0,
+      "eval_runtime": 98.4982,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 860
+    },
+    {
+      "entropy": 0.5401638634502888,
+      "epoch": 2.120627261761158,
+      "grad_norm": 0.4194582998752594,
+      "learning_rate": 6.957288653253486e-05,
+      "loss": 0.49037814140319824,
+      "mean_token_accuracy": 0.8528496414422989,
+      "num_tokens": 2149550.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.120627261761158,
+      "eval_entropy": 0.574507936835289,
+      "eval_loss": 0.5836747288703918,
+      "eval_mean_token_accuracy": 0.8325241643391298,
+      "eval_num_tokens": 2149550.0,
+      "eval_runtime": 98.5592,
+      "eval_samples_per_second": 14.408,
+      "eval_steps_per_second": 1.806,
+      "step": 880
+    },
+    {
+      "entropy": 0.5606910541653634,
+      "epoch": 2.168878166465621,
+      "grad_norm": 0.5395355820655823,
+      "learning_rate": 6.933683409127401e-05,
+      "loss": 0.5089571475982666,
+      "mean_token_accuracy": 0.8459660410881042,
+      "num_tokens": 2202162.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.168878166465621,
+      "eval_entropy": 0.5779740023479033,
+      "eval_loss": 0.5816043615341187,
+      "eval_mean_token_accuracy": 0.8326318036304431,
+      "eval_num_tokens": 2202162.0,
+      "eval_runtime": 98.4386,
+      "eval_samples_per_second": 14.425,
+      "eval_steps_per_second": 1.808,
+      "step": 900
+    },
+    {
+      "entropy": 0.5380903735756875,
+      "epoch": 2.2171290711700844,
+      "grad_norm": 0.46588417887687683,
+      "learning_rate": 6.909138888728257e-05,
+      "loss": 0.4902662754058838,
+      "mean_token_accuracy": 0.8493925213813782,
+      "num_tokens": 2256539.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2171290711700844,
+      "eval_entropy": 0.570154261220707,
+      "eval_loss": 0.5827520489692688,
+      "eval_mean_token_accuracy": 0.8321914167216654,
+      "eval_num_tokens": 2256539.0,
+      "eval_runtime": 98.5674,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 920
+    },
+    {
+      "entropy": 0.558555257320404,
+      "epoch": 2.2653799758745476,
+      "grad_norm": 0.44576048851013184,
+      "learning_rate": 6.883662037864996e-05,
+      "loss": 0.5000699520111084,
+      "mean_token_accuracy": 0.8475695163011551,
+      "num_tokens": 2299698.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2653799758745476,
+      "eval_entropy": 0.5715417749761196,
+      "eval_loss": 0.5831062197685242,
+      "eval_mean_token_accuracy": 0.8327409899636601,
+      "eval_num_tokens": 2299698.0,
+      "eval_runtime": 98.568,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 940
+    },
+    {
+      "entropy": 0.5582522705197335,
+      "epoch": 2.313630880579011,
+      "grad_norm": 0.38966405391693115,
+      "learning_rate": 6.857260066185045e-05,
+      "loss": 0.5132341861724854,
+      "mean_token_accuracy": 0.8466864034533501,
+      "num_tokens": 2347134.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.313630880579011,
+      "eval_entropy": 0.5823918579334624,
+      "eval_loss": 0.5797588229179382,
+      "eval_mean_token_accuracy": 0.8326037566313583,
+      "eval_num_tokens": 2347134.0,
+      "eval_runtime": 98.6222,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 960
+    },
+    {
+      "entropy": 0.5557353265583516,
+      "epoch": 2.361881785283474,
+      "grad_norm": 0.42300811409950256,
+      "learning_rate": 6.829940445134089e-05,
+      "loss": 0.5035375118255615,
+      "mean_token_accuracy": 0.8479988545179367,
+      "num_tokens": 2394568.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.361881785283474,
+      "eval_entropy": 0.5542296482605881,
+      "eval_loss": 0.5800604820251465,
+      "eval_mean_token_accuracy": 0.8336404163515969,
+      "eval_num_tokens": 2394568.0,
+      "eval_runtime": 98.615,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 980
+    },
+    {
+      "entropy": 0.5492161035537719,
+      "epoch": 2.4101326899879374,
+      "grad_norm": 0.47482752799987793,
+      "learning_rate": 6.801710905841724e-05,
+      "loss": 0.5032052040100098,
+      "mean_token_accuracy": 0.8488596931099892,
+      "num_tokens": 2443253.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4101326899879374,
+      "eval_entropy": 0.5755258423558781,
+      "eval_loss": 0.5764774680137634,
+      "eval_mean_token_accuracy": 0.8336571763070781,
+      "eval_num_tokens": 2443253.0,
+      "eval_runtime": 98.4765,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5439704649150372,
+      "epoch": 2.4583835946924006,
+      "grad_norm": 0.3656543493270874,
+      "learning_rate": 6.772579436933644e-05,
+      "loss": 0.49903340339660646,
+      "mean_token_accuracy": 0.8499931409955025,
+      "num_tokens": 2495178.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4583835946924006,
+      "eval_entropy": 0.5712662409530597,
+      "eval_loss": 0.5777108073234558,
+      "eval_mean_token_accuracy": 0.8336941352051296,
+      "eval_num_tokens": 2495178.0,
+      "eval_runtime": 98.678,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 1020
+    },
+    {
+      "entropy": 0.5448883637785912,
+      "epoch": 2.5066344993968634,
+      "grad_norm": 0.3990268111228943,
+      "learning_rate": 6.742554282270959e-05,
+      "loss": 0.4990490436553955,
+      "mean_token_accuracy": 0.8488436549901962,
+      "num_tokens": 2541733.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5066344993968634,
+      "eval_entropy": 0.5816385483139017,
+      "eval_loss": 0.5749375820159912,
+      "eval_mean_token_accuracy": 0.833801462074344,
+      "eval_num_tokens": 2541733.0,
+      "eval_runtime": 98.6209,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 1040
+    },
+    {
+      "entropy": 0.5575303137302399,
+      "epoch": 2.554885404101327,
+      "grad_norm": 0.45015832781791687,
+      "learning_rate": 6.711643938617279e-05,
+      "loss": 0.5061063289642334,
+      "mean_token_accuracy": 0.8462719902396202,
+      "num_tokens": 2589479.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.554885404101327,
+      "eval_entropy": 0.5666238901655326,
+      "eval_loss": 0.574820876121521,
+      "eval_mean_token_accuracy": 0.8339746617869045,
+      "eval_num_tokens": 2589479.0,
+      "eval_runtime": 98.5332,
+      "eval_samples_per_second": 14.411,
+      "eval_steps_per_second": 1.806,
+      "step": 1060
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.370774678528e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1080/trainer_state.json

@@ -0,0 +1,1168 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.60313630880579,
+  "eval_steps": 20,
+  "global_step": 1080,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    },
+    {
+      "entropy": 0.8518766984343529,
+      "epoch": 0.28950542822677927,
+      "grad_norm": 0.635079026222229,
+      "learning_rate": 2.0729210702058663e-05,
+      "loss": 0.8058656692504883,
+      "mean_token_accuracy": 0.7847469851374627,
+      "num_tokens": 298041.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28950542822677927,
+      "eval_entropy": 0.838603059227547,
+      "eval_loss": 0.7992516756057739,
+      "eval_mean_token_accuracy": 0.7849764954508021,
+      "eval_num_tokens": 298041.0,
+      "eval_runtime": 98.5627,
+      "eval_samples_per_second": 14.407,
+      "eval_steps_per_second": 1.806,
+      "step": 120
+    },
+    {
+      "entropy": 0.8069365546107292,
+      "epoch": 0.33775633293124246,
+      "grad_norm": 0.5280472636222839,
+      "learning_rate": 2.421311166038785e-05,
+      "loss": 0.7562496662139893,
+      "mean_token_accuracy": 0.7897997036576271,
+      "num_tokens": 348955.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33775633293124246,
+      "eval_entropy": 0.8161682168419442,
+      "eval_loss": 0.7599642276763916,
+      "eval_mean_token_accuracy": 0.7912511644738444,
+      "eval_num_tokens": 348955.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 140
+    },
+    {
+      "entropy": 0.7952415809035301,
+      "epoch": 0.38600723763570566,
+      "grad_norm": 0.5387083888053894,
+      "learning_rate": 2.7697012618717034e-05,
+      "loss": 0.740808629989624,
+      "mean_token_accuracy": 0.7941806614398956,
+      "num_tokens": 395798.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38600723763570566,
+      "eval_entropy": 0.7856516191798649,
+      "eval_loss": 0.7324908971786499,
+      "eval_mean_token_accuracy": 0.7980317825011993,
+      "eval_num_tokens": 395798.0,
+      "eval_runtime": 98.6378,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 160
+    },
+    {
+      "entropy": 0.7537026956677437,
+      "epoch": 0.43425814234016885,
+      "grad_norm": 0.5521425604820251,
+      "learning_rate": 3.118091357704622e-05,
+      "loss": 0.7073616027832031,
+      "mean_token_accuracy": 0.8006261065602303,
+      "num_tokens": 446437.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43425814234016885,
+      "eval_entropy": 0.7470693678668375,
+      "eval_loss": 0.711143434047699,
+      "eval_mean_token_accuracy": 0.801459076029531,
+      "eval_num_tokens": 446437.0,
+      "eval_runtime": 98.6667,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 180
+    },
+    {
+      "entropy": 0.7328549668192863,
+      "epoch": 0.4825090470446321,
+      "grad_norm": 0.4901895821094513,
+      "learning_rate": 3.4664814535375406e-05,
+      "loss": 0.6874266624450683,
+      "mean_token_accuracy": 0.8066758885979652,
+      "num_tokens": 497682.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4825090470446321,
+      "eval_entropy": 0.7374835154983435,
+      "eval_loss": 0.6967977285385132,
+      "eval_mean_token_accuracy": 0.8055616520094068,
+      "eval_num_tokens": 497682.0,
+      "eval_runtime": 98.6234,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 200
+    },
+    {
+      "entropy": 0.7372006759047508,
+      "epoch": 0.5307599517490953,
+      "grad_norm": 0.5516473650932312,
+      "learning_rate": 3.814871549370459e-05,
+      "loss": 0.6906380653381348,
+      "mean_token_accuracy": 0.8057604253292083,
+      "num_tokens": 544670.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5307599517490953,
+      "eval_entropy": 0.710068748573239,
+      "eval_loss": 0.6860950589179993,
+      "eval_mean_token_accuracy": 0.8069355440943429,
+      "eval_num_tokens": 544670.0,
+      "eval_runtime": 98.4618,
+      "eval_samples_per_second": 14.422,
+      "eval_steps_per_second": 1.808,
+      "step": 220
+    },
+    {
+      "entropy": 0.7099859565496445,
+      "epoch": 0.5790108564535585,
+      "grad_norm": 0.457378625869751,
+      "learning_rate": 4.1632616452033785e-05,
+      "loss": 0.6674468517303467,
+      "mean_token_accuracy": 0.8087703332304954,
+      "num_tokens": 592926.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5790108564535585,
+      "eval_entropy": 0.717441116156203,
+      "eval_loss": 0.6765118837356567,
+      "eval_mean_token_accuracy": 0.8095786246690857,
+      "eval_num_tokens": 592926.0,
+      "eval_runtime": 98.6828,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 240
+    },
+    {
+      "entropy": 0.7170855522155761,
+      "epoch": 0.6272617611580217,
+      "grad_norm": 0.4604854881763458,
+      "learning_rate": 4.5116517410362974e-05,
+      "loss": 0.6647133827209473,
+      "mean_token_accuracy": 0.8115006521344185,
+      "num_tokens": 639077.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6272617611580217,
+      "eval_entropy": 0.7082702219486237,
+      "eval_loss": 0.6722468733787537,
+      "eval_mean_token_accuracy": 0.8100237367528208,
+      "eval_num_tokens": 639077.0,
+      "eval_runtime": 98.6242,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 260
+    },
+    {
+      "entropy": 0.691286925971508,
+      "epoch": 0.6755126658624849,
+      "grad_norm": 0.4626481831073761,
+      "learning_rate": 4.860041836869215e-05,
+      "loss": 0.6543298721313476,
+      "mean_token_accuracy": 0.8142751976847649,
+      "num_tokens": 691028.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6755126658624849,
+      "eval_entropy": 0.7341568747932992,
+      "eval_loss": 0.658997654914856,
+      "eval_mean_token_accuracy": 0.813439255684949,
+      "eval_num_tokens": 691028.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 280
+    },
+    {
+      "entropy": 0.691513468325138,
+      "epoch": 0.7237635705669482,
+      "grad_norm": 0.5434145331382751,
+      "learning_rate": 5.2084319327021346e-05,
+      "loss": 0.6393396377563476,
+      "mean_token_accuracy": 0.8172704800963402,
+      "num_tokens": 740255.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7237635705669482,
+      "eval_entropy": 0.699160492487168,
+      "eval_loss": 0.6522852182388306,
+      "eval_mean_token_accuracy": 0.8143363323774231,
+      "eval_num_tokens": 740255.0,
+      "eval_runtime": 98.6233,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 300
+    },
+    {
+      "entropy": 0.6918429866433143,
+      "epoch": 0.7720144752714113,
+      "grad_norm": 0.47131890058517456,
+      "learning_rate": 5.556822028535052e-05,
+      "loss": 0.6503581523895263,
+      "mean_token_accuracy": 0.8149650946259499,
+      "num_tokens": 789529.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7720144752714113,
+      "eval_entropy": 0.6949931701247611,
+      "eval_loss": 0.6464414596557617,
+      "eval_mean_token_accuracy": 0.816310279489903,
+      "eval_num_tokens": 789529.0,
+      "eval_runtime": 98.4759,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 320
+    },
+    {
+      "entropy": 0.6926439955830574,
+      "epoch": 0.8202653799758746,
+      "grad_norm": 0.46790942549705505,
+      "learning_rate": 5.905212124367972e-05,
+      "loss": 0.6453651428222656,
+      "mean_token_accuracy": 0.8171535924077034,
+      "num_tokens": 841584.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8202653799758746,
+      "eval_entropy": 0.6810062698433909,
+      "eval_loss": 0.6440736651420593,
+      "eval_mean_token_accuracy": 0.8166696479481258,
+      "eval_num_tokens": 841584.0,
+      "eval_runtime": 98.6479,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 340
+    },
+    {
+      "entropy": 0.7038845136761666,
+      "epoch": 0.8685162846803377,
+      "grad_norm": 0.5038695335388184,
+      "learning_rate": 6.253602220200891e-05,
+      "loss": 0.6558614253997803,
+      "mean_token_accuracy": 0.8127013072371483,
+      "num_tokens": 885769.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8685162846803377,
+      "eval_entropy": 0.6752923710292644,
+      "eval_loss": 0.6410524845123291,
+      "eval_mean_token_accuracy": 0.817534717950928,
+      "eval_num_tokens": 885769.0,
+      "eval_runtime": 98.5986,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 360
+    },
+    {
+      "entropy": 0.6897840186953544,
+      "epoch": 0.916767189384801,
+      "grad_norm": 0.5168097615242004,
+      "learning_rate": 6.601992316033808e-05,
+      "loss": 0.6328047752380371,
+      "mean_token_accuracy": 0.8181755512952804,
+      "num_tokens": 930056.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.916767189384801,
+      "eval_entropy": 0.6816390936964014,
+      "eval_loss": 0.6402509212493896,
+      "eval_mean_token_accuracy": 0.8172605586855599,
+      "eval_num_tokens": 930056.0,
+      "eval_runtime": 98.4787,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 380
+    },
+    {
+      "entropy": 0.6798740655183793,
+      "epoch": 0.9650180940892642,
+      "grad_norm": 0.4609147906303406,
+      "learning_rate": 6.950382411866727e-05,
+      "loss": 0.6358067989349365,
+      "mean_token_accuracy": 0.8193414479494094,
+      "num_tokens": 976892.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9650180940892642,
+      "eval_entropy": 0.6841439577970612,
+      "eval_loss": 0.6335230469703674,
+      "eval_mean_token_accuracy": 0.8190797965848045,
+      "eval_num_tokens": 976892.0,
+      "eval_runtime": 98.5983,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 400
+    },
+    {
+      "entropy": 0.6647876103719076,
+      "epoch": 1.0120627261761157,
+      "grad_norm": 0.46518462896347046,
+      "learning_rate": 7.229074030587624e-05,
+      "loss": 0.6112009525299072,
+      "mean_token_accuracy": 0.8212849283829714,
+      "num_tokens": 1024472.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0120627261761157,
+      "eval_entropy": 0.6612423688508151,
+      "eval_loss": 0.6311497688293457,
+      "eval_mean_token_accuracy": 0.8194578638237514,
+      "eval_num_tokens": 1024472.0,
+      "eval_runtime": 98.4872,
+      "eval_samples_per_second": 14.418,
+      "eval_steps_per_second": 1.807,
+      "step": 420
+    },
+    {
+      "entropy": 0.6288895070552826,
+      "epoch": 1.060313630880579,
+      "grad_norm": 0.4342021644115448,
+      "learning_rate": 7.228358026812833e-05,
+      "loss": 0.5853999614715576,
+      "mean_token_accuracy": 0.82901301831007,
+      "num_tokens": 1076964.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060313630880579,
+      "eval_entropy": 0.6567474329069759,
+      "eval_loss": 0.6245927810668945,
+      "eval_mean_token_accuracy": 0.8211770050981072,
+      "eval_num_tokens": 1076964.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 440
+    },
+    {
+      "entropy": 0.639747454226017,
+      "epoch": 1.1085645355850422,
+      "grad_norm": 0.5338298082351685,
+      "learning_rate": 7.226619357333845e-05,
+      "loss": 0.5847910404205322,
+      "mean_token_accuracy": 0.8279331669211387,
+      "num_tokens": 1126064.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1085645355850422,
+      "eval_entropy": 0.6399250211340658,
+      "eval_loss": 0.6245712637901306,
+      "eval_mean_token_accuracy": 0.8209733875949731,
+      "eval_num_tokens": 1126064.0,
+      "eval_runtime": 98.6581,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 460
+    },
+    {
+      "entropy": 0.6248803421854973,
+      "epoch": 1.1568154402895054,
+      "grad_norm": 0.4402505159378052,
+      "learning_rate": 7.223858514173552e-05,
+      "loss": 0.5847790241241455,
+      "mean_token_accuracy": 0.8273933246731758,
+      "num_tokens": 1176912.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1568154402895054,
+      "eval_entropy": 0.6459465750147787,
+      "eval_loss": 0.615716278553009,
+      "eval_mean_token_accuracy": 0.8228987813665626,
+      "eval_num_tokens": 1176912.0,
+      "eval_runtime": 98.5987,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 480
+    },
+    {
+      "entropy": 0.6356995061039925,
+      "epoch": 1.2050663449939687,
+      "grad_norm": 0.5318694114685059,
+      "learning_rate": 7.220076278617914e-05,
+      "loss": 0.5902805328369141,
+      "mean_token_accuracy": 0.8279137581586837,
+      "num_tokens": 1223383.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2050663449939687,
+      "eval_entropy": 0.6509443488013879,
+      "eval_loss": 0.6174858808517456,
+      "eval_mean_token_accuracy": 0.8229377031996009,
+      "eval_num_tokens": 1223383.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 500
+    },
+    {
+      "entropy": 0.6378119193017483,
+      "epoch": 1.2533172496984317,
+      "grad_norm": 0.5385730266571045,
+      "learning_rate": 7.215273720994853e-05,
+      "loss": 0.5850183486938476,
+      "mean_token_accuracy": 0.8282771646976471,
+      "num_tokens": 1273534.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2533172496984317,
+      "eval_entropy": 0.6438969477508845,
+      "eval_loss": 0.6134508848190308,
+      "eval_mean_token_accuracy": 0.823984161186754,
+      "eval_num_tokens": 1273534.0,
+      "eval_runtime": 98.4814,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 520
+    },
+    {
+      "entropy": 0.6330174207687378,
+      "epoch": 1.301568154402895,
+      "grad_norm": 0.4950766861438751,
+      "learning_rate": 7.209452200371367e-05,
+      "loss": 0.5886627674102783,
+      "mean_token_accuracy": 0.828557687997818,
+      "num_tokens": 1323351.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.301568154402895,
+      "eval_entropy": 0.6342737025089478,
+      "eval_loss": 0.611145555973053,
+      "eval_mean_token_accuracy": 0.8250270244780551,
+      "eval_num_tokens": 1323351.0,
+      "eval_runtime": 98.6698,
+      "eval_samples_per_second": 14.391,
+      "eval_steps_per_second": 1.804,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270076125860214,
+      "epoch": 1.3498190591073582,
+      "grad_norm": 0.435604989528656,
+      "learning_rate": 7.20261336416893e-05,
+      "loss": 0.5834732532501221,
+      "mean_token_accuracy": 0.8290600389242172,
+      "num_tokens": 1371508.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3498190591073582,
+      "eval_entropy": 0.6624904408883513,
+      "eval_loss": 0.6063718795776367,
+      "eval_mean_token_accuracy": 0.825260305002834,
+      "eval_num_tokens": 1371508.0,
+      "eval_runtime": 98.6348,
+      "eval_samples_per_second": 14.397,
+      "eval_steps_per_second": 1.805,
+      "step": 560
+    },
+    {
+      "entropy": 0.6258987173438072,
+      "epoch": 1.3980699638118215,
+      "grad_norm": 0.45945441722869873,
+      "learning_rate": 7.194759147697293e-05,
+      "loss": 0.5705549716949463,
+      "mean_token_accuracy": 0.8329991281032563,
+      "num_tokens": 1415221.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3980699638118215,
+      "eval_entropy": 0.6500632069753797,
+      "eval_loss": 0.6059838533401489,
+      "eval_mean_token_accuracy": 0.8262855604793249,
+      "eval_num_tokens": 1415221.0,
+      "eval_runtime": 98.6469,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 580
+    },
+    {
+      "entropy": 0.6301489062607288,
+      "epoch": 1.4463208685162847,
+      "grad_norm": 0.5217350721359253,
+      "learning_rate": 7.18589177360681e-05,
+      "loss": 0.5778427124023438,
+      "mean_token_accuracy": 0.8305584967136384,
+      "num_tokens": 1463356.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4463208685162847,
+      "eval_entropy": 0.6478241879618569,
+      "eval_loss": 0.6014655828475952,
+      "eval_mean_token_accuracy": 0.8265940369514937,
+      "eval_num_tokens": 1463356.0,
+      "eval_runtime": 98.6523,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 600
+    },
+    {
+      "entropy": 0.6257719069719314,
+      "epoch": 1.494571773220748,
+      "grad_norm": 0.48310142755508423,
+      "learning_rate": 7.176013751259459e-05,
+      "loss": 0.5713310241699219,
+      "mean_token_accuracy": 0.8305533796548843,
+      "num_tokens": 1512340.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.494571773220748,
+      "eval_entropy": 0.6396401956510008,
+      "eval_loss": 0.5978795289993286,
+      "eval_mean_token_accuracy": 0.8273170630583603,
+      "eval_num_tokens": 1512340.0,
+      "eval_runtime": 98.4958,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 620
+    },
+    {
+      "entropy": 0.6162159442901611,
+      "epoch": 1.5428226779252112,
+      "grad_norm": 0.4500057101249695,
+      "learning_rate": 7.165127876018717e-05,
+      "loss": 0.565293264389038,
+      "mean_token_accuracy": 0.8311659932136536,
+      "num_tokens": 1560911.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5428226779252112,
+      "eval_entropy": 0.6381519051750054,
+      "eval_loss": 0.5963984131813049,
+      "eval_mean_token_accuracy": 0.8268409247478742,
+      "eval_num_tokens": 1560911.0,
+      "eval_runtime": 98.6803,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 640
+    },
+    {
+      "entropy": 0.6088747605681419,
+      "epoch": 1.5910735826296745,
+      "grad_norm": 0.46954667568206787,
+      "learning_rate": 7.153237228458511e-05,
+      "loss": 0.5602522850036621,
+      "mean_token_accuracy": 0.8345131903886795,
+      "num_tokens": 1609409.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5910735826296745,
+      "eval_entropy": 0.6413098875056492,
+      "eval_loss": 0.5938988327980042,
+      "eval_mean_token_accuracy": 0.828771137119679,
+      "eval_num_tokens": 1609409.0,
+      "eval_runtime": 98.4951,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 660
+    },
+    {
+      "entropy": 0.6209868296980858,
+      "epoch": 1.6393244873341375,
+      "grad_norm": 0.3867878317832947,
+      "learning_rate": 7.140345173491446e-05,
+      "loss": 0.5663797378540039,
+      "mean_token_accuracy": 0.8352224260568619,
+      "num_tokens": 1657430.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.6393244873341375,
+      "eval_entropy": 0.6182791748073664,
+      "eval_loss": 0.5919068455696106,
+      "eval_mean_token_accuracy": 0.8286421476455217,
+      "eval_num_tokens": 1657430.0,
+      "eval_runtime": 98.6211,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 680
+    },
+    {
+      "entropy": 0.6191923573613167,
+      "epoch": 1.6875753920386007,
+      "grad_norm": 0.40004226565361023,
+      "learning_rate": 7.12645535941658e-05,
+      "loss": 0.57517991065979,
+      "mean_token_accuracy": 0.8333286643028259,
+      "num_tokens": 1708058.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.6875753920386007,
+      "eval_entropy": 0.6244879199212856,
+      "eval_loss": 0.5897895693778992,
+      "eval_mean_token_accuracy": 0.8293621252092083,
+      "eval_num_tokens": 1708058.0,
+      "eval_runtime": 98.4974,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 700
+    },
+    {
+      "entropy": 0.6199282392859459,
+      "epoch": 1.7358262967430638,
+      "grad_norm": 0.36575791239738464,
+      "learning_rate": 7.111571716886997e-05,
+      "loss": 0.5717356681823731,
+      "mean_token_accuracy": 0.8321028783917427,
+      "num_tokens": 1761262.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.7358262967430638,
+      "eval_entropy": 0.6162440127201294,
+      "eval_loss": 0.5864423513412476,
+      "eval_mean_token_accuracy": 0.830874237786518,
+      "eval_num_tokens": 1761262.0,
+      "eval_runtime": 98.6109,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 720
+    },
+    {
+      "entropy": 0.6127455830574036,
+      "epoch": 1.784077201447527,
+      "grad_norm": 0.4778473973274231,
+      "learning_rate": 7.095698457797475e-05,
+      "loss": 0.5664590835571289,
+      "mean_token_accuracy": 0.8324130088090896,
+      "num_tokens": 1812127.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.784077201447527,
+      "eval_entropy": 0.6273092945640006,
+      "eval_loss": 0.5847376585006714,
+      "eval_mean_token_accuracy": 0.831051754817534,
+      "eval_num_tokens": 1812127.0,
+      "eval_runtime": 98.6513,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 740
+    },
+    {
+      "entropy": 0.6141144648194313,
+      "epoch": 1.8323281061519903,
+      "grad_norm": 0.6467089056968689,
+      "learning_rate": 7.078840074092574e-05,
+      "loss": 0.5548617362976074,
+      "mean_token_accuracy": 0.8342376738786698,
+      "num_tokens": 1858403.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.8323281061519903,
+      "eval_entropy": 0.5951976082967908,
+      "eval_loss": 0.5850171446800232,
+      "eval_mean_token_accuracy": 0.8303041870004675,
+      "eval_num_tokens": 1858403.0,
+      "eval_runtime": 98.6601,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 760
+    },
+    {
+      "entropy": 0.5961867064237595,
+      "epoch": 1.8805790108564535,
+      "grad_norm": 0.41280242800712585,
+      "learning_rate": 7.061001336495465e-05,
+      "loss": 0.5511267185211182,
+      "mean_token_accuracy": 0.8392465204000473,
+      "num_tokens": 1907361.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8805790108564535,
+      "eval_entropy": 0.6167017998320333,
+      "eval_loss": 0.5826791524887085,
+      "eval_mean_token_accuracy": 0.8309678050239434,
+      "eval_num_tokens": 1907361.0,
+      "eval_runtime": 98.6564,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 780
+    },
+    {
+      "entropy": 0.6025628983974457,
+      "epoch": 1.9288299155609168,
+      "grad_norm": 0.4078947901725769,
+      "learning_rate": 7.042187293157881e-05,
+      "loss": 0.5552026271820069,
+      "mean_token_accuracy": 0.8359778299927711,
+      "num_tokens": 1956605.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9288299155609168,
+      "eval_entropy": 0.6047008484601974,
+      "eval_loss": 0.5811123847961426,
+      "eval_mean_token_accuracy": 0.8320730804057603,
+      "eval_num_tokens": 1956605.0,
+      "eval_runtime": 98.6033,
+      "eval_samples_per_second": 14.401,
+      "eval_steps_per_second": 1.805,
+      "step": 800
+    },
+    {
+      "entropy": 0.6041995719075203,
+      "epoch": 1.97708082026538,
+      "grad_norm": 0.543890118598938,
+      "learning_rate": 7.022403268231535e-05,
+      "loss": 0.554384422302246,
+      "mean_token_accuracy": 0.8380729153752327,
+      "num_tokens": 2003308.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.97708082026538,
+      "eval_entropy": 0.6005127917179901,
+      "eval_loss": 0.5800131559371948,
+      "eval_mean_token_accuracy": 0.8313186547059691,
+      "eval_num_tokens": 2003308.0,
+      "eval_runtime": 98.4119,
+      "eval_samples_per_second": 14.429,
+      "eval_steps_per_second": 1.809,
+      "step": 820
+    },
+    {
+      "entropy": 0.567442889396961,
+      "epoch": 2.0241254523522314,
+      "grad_norm": 0.5015465021133423,
+      "learning_rate": 7.001654860361456e-05,
+      "loss": 0.5208251476287842,
+      "mean_token_accuracy": 0.8442784960453327,
+      "num_tokens": 2051331.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0241254523522314,
+      "eval_entropy": 0.5785037809878253,
+      "eval_loss": 0.5865591168403625,
+      "eval_mean_token_accuracy": 0.8307993857378371,
+      "eval_num_tokens": 2051331.0,
+      "eval_runtime": 98.5061,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 840
+    },
+    {
+      "entropy": 0.5552012033760547,
+      "epoch": 2.0723763570566947,
+      "grad_norm": 0.5975943207740784,
+      "learning_rate": 6.979947941101646e-05,
+      "loss": 0.5064489841461182,
+      "mean_token_accuracy": 0.8468396887183189,
+      "num_tokens": 2101528.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0723763570566947,
+      "eval_entropy": 0.5844205239515626,
+      "eval_loss": 0.585284948348999,
+      "eval_mean_token_accuracy": 0.8309797065311604,
+      "eval_num_tokens": 2101528.0,
+      "eval_runtime": 98.4982,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 860
+    },
+    {
+      "entropy": 0.5401638634502888,
+      "epoch": 2.120627261761158,
+      "grad_norm": 0.4194582998752594,
+      "learning_rate": 6.957288653253486e-05,
+      "loss": 0.49037814140319824,
+      "mean_token_accuracy": 0.8528496414422989,
+      "num_tokens": 2149550.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.120627261761158,
+      "eval_entropy": 0.574507936835289,
+      "eval_loss": 0.5836747288703918,
+      "eval_mean_token_accuracy": 0.8325241643391298,
+      "eval_num_tokens": 2149550.0,
+      "eval_runtime": 98.5592,
+      "eval_samples_per_second": 14.408,
+      "eval_steps_per_second": 1.806,
+      "step": 880
+    },
+    {
+      "entropy": 0.5606910541653634,
+      "epoch": 2.168878166465621,
+      "grad_norm": 0.5395355820655823,
+      "learning_rate": 6.933683409127401e-05,
+      "loss": 0.5089571475982666,
+      "mean_token_accuracy": 0.8459660410881042,
+      "num_tokens": 2202162.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.168878166465621,
+      "eval_entropy": 0.5779740023479033,
+      "eval_loss": 0.5816043615341187,
+      "eval_mean_token_accuracy": 0.8326318036304431,
+      "eval_num_tokens": 2202162.0,
+      "eval_runtime": 98.4386,
+      "eval_samples_per_second": 14.425,
+      "eval_steps_per_second": 1.808,
+      "step": 900
+    },
+    {
+      "entropy": 0.5380903735756875,
+      "epoch": 2.2171290711700844,
+      "grad_norm": 0.46588417887687683,
+      "learning_rate": 6.909138888728257e-05,
+      "loss": 0.4902662754058838,
+      "mean_token_accuracy": 0.8493925213813782,
+      "num_tokens": 2256539.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2171290711700844,
+      "eval_entropy": 0.570154261220707,
+      "eval_loss": 0.5827520489692688,
+      "eval_mean_token_accuracy": 0.8321914167216654,
+      "eval_num_tokens": 2256539.0,
+      "eval_runtime": 98.5674,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 920
+    },
+    {
+      "entropy": 0.558555257320404,
+      "epoch": 2.2653799758745476,
+      "grad_norm": 0.44576048851013184,
+      "learning_rate": 6.883662037864996e-05,
+      "loss": 0.5000699520111084,
+      "mean_token_accuracy": 0.8475695163011551,
+      "num_tokens": 2299698.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2653799758745476,
+      "eval_entropy": 0.5715417749761196,
+      "eval_loss": 0.5831062197685242,
+      "eval_mean_token_accuracy": 0.8327409899636601,
+      "eval_num_tokens": 2299698.0,
+      "eval_runtime": 98.568,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 940
+    },
+    {
+      "entropy": 0.5582522705197335,
+      "epoch": 2.313630880579011,
+      "grad_norm": 0.38966405391693115,
+      "learning_rate": 6.857260066185045e-05,
+      "loss": 0.5132341861724854,
+      "mean_token_accuracy": 0.8466864034533501,
+      "num_tokens": 2347134.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.313630880579011,
+      "eval_entropy": 0.5823918579334624,
+      "eval_loss": 0.5797588229179382,
+      "eval_mean_token_accuracy": 0.8326037566313583,
+      "eval_num_tokens": 2347134.0,
+      "eval_runtime": 98.6222,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 960
+    },
+    {
+      "entropy": 0.5557353265583516,
+      "epoch": 2.361881785283474,
+      "grad_norm": 0.42300811409950256,
+      "learning_rate": 6.829940445134089e-05,
+      "loss": 0.5035375118255615,
+      "mean_token_accuracy": 0.8479988545179367,
+      "num_tokens": 2394568.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.361881785283474,
+      "eval_entropy": 0.5542296482605881,
+      "eval_loss": 0.5800604820251465,
+      "eval_mean_token_accuracy": 0.8336404163515969,
+      "eval_num_tokens": 2394568.0,
+      "eval_runtime": 98.615,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 980
+    },
+    {
+      "entropy": 0.5492161035537719,
+      "epoch": 2.4101326899879374,
+      "grad_norm": 0.47482752799987793,
+      "learning_rate": 6.801710905841724e-05,
+      "loss": 0.5032052040100098,
+      "mean_token_accuracy": 0.8488596931099892,
+      "num_tokens": 2443253.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4101326899879374,
+      "eval_entropy": 0.5755258423558781,
+      "eval_loss": 0.5764774680137634,
+      "eval_mean_token_accuracy": 0.8336571763070781,
+      "eval_num_tokens": 2443253.0,
+      "eval_runtime": 98.4765,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5439704649150372,
+      "epoch": 2.4583835946924006,
+      "grad_norm": 0.3656543493270874,
+      "learning_rate": 6.772579436933644e-05,
+      "loss": 0.49903340339660646,
+      "mean_token_accuracy": 0.8499931409955025,
+      "num_tokens": 2495178.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4583835946924006,
+      "eval_entropy": 0.5712662409530597,
+      "eval_loss": 0.5777108073234558,
+      "eval_mean_token_accuracy": 0.8336941352051296,
+      "eval_num_tokens": 2495178.0,
+      "eval_runtime": 98.678,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 1020
+    },
+    {
+      "entropy": 0.5448883637785912,
+      "epoch": 2.5066344993968634,
+      "grad_norm": 0.3990268111228943,
+      "learning_rate": 6.742554282270959e-05,
+      "loss": 0.4990490436553955,
+      "mean_token_accuracy": 0.8488436549901962,
+      "num_tokens": 2541733.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5066344993968634,
+      "eval_entropy": 0.5816385483139017,
+      "eval_loss": 0.5749375820159912,
+      "eval_mean_token_accuracy": 0.833801462074344,
+      "eval_num_tokens": 2541733.0,
+      "eval_runtime": 98.6209,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 1040
+    },
+    {
+      "entropy": 0.5575303137302399,
+      "epoch": 2.554885404101327,
+      "grad_norm": 0.45015832781791687,
+      "learning_rate": 6.711643938617279e-05,
+      "loss": 0.5061063289642334,
+      "mean_token_accuracy": 0.8462719902396202,
+      "num_tokens": 2589479.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.554885404101327,
+      "eval_entropy": 0.5666238901655326,
+      "eval_loss": 0.574820876121521,
+      "eval_mean_token_accuracy": 0.8339746617869045,
+      "eval_num_tokens": 2589479.0,
+      "eval_runtime": 98.5332,
+      "eval_samples_per_second": 14.411,
+      "eval_steps_per_second": 1.806,
+      "step": 1060
+    },
+    {
+      "entropy": 0.5477629877626896,
+      "epoch": 2.60313630880579,
+      "grad_norm": 0.43583670258522034,
+      "learning_rate": 6.679857153234218e-05,
+      "loss": 0.4998013496398926,
+      "mean_token_accuracy": 0.8465577483177185,
+      "num_tokens": 2639134.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.60313630880579,
+      "eval_entropy": 0.5809191281206152,
+      "eval_loss": 0.5737497210502625,
+      "eval_mean_token_accuracy": 0.8338909467284599,
+      "eval_num_tokens": 2639134.0,
+      "eval_runtime": 98.5093,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 1080
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.47206953373696e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1100/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1100/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1100/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1100/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1100/trainer_state.json

@@ -0,0 +1,1189 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.651387213510253,
+  "eval_steps": 20,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    },
+    {
+      "entropy": 0.8518766984343529,
+      "epoch": 0.28950542822677927,
+      "grad_norm": 0.635079026222229,
+      "learning_rate": 2.0729210702058663e-05,
+      "loss": 0.8058656692504883,
+      "mean_token_accuracy": 0.7847469851374627,
+      "num_tokens": 298041.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28950542822677927,
+      "eval_entropy": 0.838603059227547,
+      "eval_loss": 0.7992516756057739,
+      "eval_mean_token_accuracy": 0.7849764954508021,
+      "eval_num_tokens": 298041.0,
+      "eval_runtime": 98.5627,
+      "eval_samples_per_second": 14.407,
+      "eval_steps_per_second": 1.806,
+      "step": 120
+    },
+    {
+      "entropy": 0.8069365546107292,
+      "epoch": 0.33775633293124246,
+      "grad_norm": 0.5280472636222839,
+      "learning_rate": 2.421311166038785e-05,
+      "loss": 0.7562496662139893,
+      "mean_token_accuracy": 0.7897997036576271,
+      "num_tokens": 348955.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33775633293124246,
+      "eval_entropy": 0.8161682168419442,
+      "eval_loss": 0.7599642276763916,
+      "eval_mean_token_accuracy": 0.7912511644738444,
+      "eval_num_tokens": 348955.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 140
+    },
+    {
+      "entropy": 0.7952415809035301,
+      "epoch": 0.38600723763570566,
+      "grad_norm": 0.5387083888053894,
+      "learning_rate": 2.7697012618717034e-05,
+      "loss": 0.740808629989624,
+      "mean_token_accuracy": 0.7941806614398956,
+      "num_tokens": 395798.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38600723763570566,
+      "eval_entropy": 0.7856516191798649,
+      "eval_loss": 0.7324908971786499,
+      "eval_mean_token_accuracy": 0.7980317825011993,
+      "eval_num_tokens": 395798.0,
+      "eval_runtime": 98.6378,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 160
+    },
+    {
+      "entropy": 0.7537026956677437,
+      "epoch": 0.43425814234016885,
+      "grad_norm": 0.5521425604820251,
+      "learning_rate": 3.118091357704622e-05,
+      "loss": 0.7073616027832031,
+      "mean_token_accuracy": 0.8006261065602303,
+      "num_tokens": 446437.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43425814234016885,
+      "eval_entropy": 0.7470693678668375,
+      "eval_loss": 0.711143434047699,
+      "eval_mean_token_accuracy": 0.801459076029531,
+      "eval_num_tokens": 446437.0,
+      "eval_runtime": 98.6667,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 180
+    },
+    {
+      "entropy": 0.7328549668192863,
+      "epoch": 0.4825090470446321,
+      "grad_norm": 0.4901895821094513,
+      "learning_rate": 3.4664814535375406e-05,
+      "loss": 0.6874266624450683,
+      "mean_token_accuracy": 0.8066758885979652,
+      "num_tokens": 497682.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4825090470446321,
+      "eval_entropy": 0.7374835154983435,
+      "eval_loss": 0.6967977285385132,
+      "eval_mean_token_accuracy": 0.8055616520094068,
+      "eval_num_tokens": 497682.0,
+      "eval_runtime": 98.6234,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 200
+    },
+    {
+      "entropy": 0.7372006759047508,
+      "epoch": 0.5307599517490953,
+      "grad_norm": 0.5516473650932312,
+      "learning_rate": 3.814871549370459e-05,
+      "loss": 0.6906380653381348,
+      "mean_token_accuracy": 0.8057604253292083,
+      "num_tokens": 544670.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5307599517490953,
+      "eval_entropy": 0.710068748573239,
+      "eval_loss": 0.6860950589179993,
+      "eval_mean_token_accuracy": 0.8069355440943429,
+      "eval_num_tokens": 544670.0,
+      "eval_runtime": 98.4618,
+      "eval_samples_per_second": 14.422,
+      "eval_steps_per_second": 1.808,
+      "step": 220
+    },
+    {
+      "entropy": 0.7099859565496445,
+      "epoch": 0.5790108564535585,
+      "grad_norm": 0.457378625869751,
+      "learning_rate": 4.1632616452033785e-05,
+      "loss": 0.6674468517303467,
+      "mean_token_accuracy": 0.8087703332304954,
+      "num_tokens": 592926.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5790108564535585,
+      "eval_entropy": 0.717441116156203,
+      "eval_loss": 0.6765118837356567,
+      "eval_mean_token_accuracy": 0.8095786246690857,
+      "eval_num_tokens": 592926.0,
+      "eval_runtime": 98.6828,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 240
+    },
+    {
+      "entropy": 0.7170855522155761,
+      "epoch": 0.6272617611580217,
+      "grad_norm": 0.4604854881763458,
+      "learning_rate": 4.5116517410362974e-05,
+      "loss": 0.6647133827209473,
+      "mean_token_accuracy": 0.8115006521344185,
+      "num_tokens": 639077.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6272617611580217,
+      "eval_entropy": 0.7082702219486237,
+      "eval_loss": 0.6722468733787537,
+      "eval_mean_token_accuracy": 0.8100237367528208,
+      "eval_num_tokens": 639077.0,
+      "eval_runtime": 98.6242,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 260
+    },
+    {
+      "entropy": 0.691286925971508,
+      "epoch": 0.6755126658624849,
+      "grad_norm": 0.4626481831073761,
+      "learning_rate": 4.860041836869215e-05,
+      "loss": 0.6543298721313476,
+      "mean_token_accuracy": 0.8142751976847649,
+      "num_tokens": 691028.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6755126658624849,
+      "eval_entropy": 0.7341568747932992,
+      "eval_loss": 0.658997654914856,
+      "eval_mean_token_accuracy": 0.813439255684949,
+      "eval_num_tokens": 691028.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 280
+    },
+    {
+      "entropy": 0.691513468325138,
+      "epoch": 0.7237635705669482,
+      "grad_norm": 0.5434145331382751,
+      "learning_rate": 5.2084319327021346e-05,
+      "loss": 0.6393396377563476,
+      "mean_token_accuracy": 0.8172704800963402,
+      "num_tokens": 740255.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7237635705669482,
+      "eval_entropy": 0.699160492487168,
+      "eval_loss": 0.6522852182388306,
+      "eval_mean_token_accuracy": 0.8143363323774231,
+      "eval_num_tokens": 740255.0,
+      "eval_runtime": 98.6233,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 300
+    },
+    {
+      "entropy": 0.6918429866433143,
+      "epoch": 0.7720144752714113,
+      "grad_norm": 0.47131890058517456,
+      "learning_rate": 5.556822028535052e-05,
+      "loss": 0.6503581523895263,
+      "mean_token_accuracy": 0.8149650946259499,
+      "num_tokens": 789529.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7720144752714113,
+      "eval_entropy": 0.6949931701247611,
+      "eval_loss": 0.6464414596557617,
+      "eval_mean_token_accuracy": 0.816310279489903,
+      "eval_num_tokens": 789529.0,
+      "eval_runtime": 98.4759,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 320
+    },
+    {
+      "entropy": 0.6926439955830574,
+      "epoch": 0.8202653799758746,
+      "grad_norm": 0.46790942549705505,
+      "learning_rate": 5.905212124367972e-05,
+      "loss": 0.6453651428222656,
+      "mean_token_accuracy": 0.8171535924077034,
+      "num_tokens": 841584.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8202653799758746,
+      "eval_entropy": 0.6810062698433909,
+      "eval_loss": 0.6440736651420593,
+      "eval_mean_token_accuracy": 0.8166696479481258,
+      "eval_num_tokens": 841584.0,
+      "eval_runtime": 98.6479,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 340
+    },
+    {
+      "entropy": 0.7038845136761666,
+      "epoch": 0.8685162846803377,
+      "grad_norm": 0.5038695335388184,
+      "learning_rate": 6.253602220200891e-05,
+      "loss": 0.6558614253997803,
+      "mean_token_accuracy": 0.8127013072371483,
+      "num_tokens": 885769.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8685162846803377,
+      "eval_entropy": 0.6752923710292644,
+      "eval_loss": 0.6410524845123291,
+      "eval_mean_token_accuracy": 0.817534717950928,
+      "eval_num_tokens": 885769.0,
+      "eval_runtime": 98.5986,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 360
+    },
+    {
+      "entropy": 0.6897840186953544,
+      "epoch": 0.916767189384801,
+      "grad_norm": 0.5168097615242004,
+      "learning_rate": 6.601992316033808e-05,
+      "loss": 0.6328047752380371,
+      "mean_token_accuracy": 0.8181755512952804,
+      "num_tokens": 930056.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.916767189384801,
+      "eval_entropy": 0.6816390936964014,
+      "eval_loss": 0.6402509212493896,
+      "eval_mean_token_accuracy": 0.8172605586855599,
+      "eval_num_tokens": 930056.0,
+      "eval_runtime": 98.4787,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 380
+    },
+    {
+      "entropy": 0.6798740655183793,
+      "epoch": 0.9650180940892642,
+      "grad_norm": 0.4609147906303406,
+      "learning_rate": 6.950382411866727e-05,
+      "loss": 0.6358067989349365,
+      "mean_token_accuracy": 0.8193414479494094,
+      "num_tokens": 976892.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9650180940892642,
+      "eval_entropy": 0.6841439577970612,
+      "eval_loss": 0.6335230469703674,
+      "eval_mean_token_accuracy": 0.8190797965848045,
+      "eval_num_tokens": 976892.0,
+      "eval_runtime": 98.5983,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 400
+    },
+    {
+      "entropy": 0.6647876103719076,
+      "epoch": 1.0120627261761157,
+      "grad_norm": 0.46518462896347046,
+      "learning_rate": 7.229074030587624e-05,
+      "loss": 0.6112009525299072,
+      "mean_token_accuracy": 0.8212849283829714,
+      "num_tokens": 1024472.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0120627261761157,
+      "eval_entropy": 0.6612423688508151,
+      "eval_loss": 0.6311497688293457,
+      "eval_mean_token_accuracy": 0.8194578638237514,
+      "eval_num_tokens": 1024472.0,
+      "eval_runtime": 98.4872,
+      "eval_samples_per_second": 14.418,
+      "eval_steps_per_second": 1.807,
+      "step": 420
+    },
+    {
+      "entropy": 0.6288895070552826,
+      "epoch": 1.060313630880579,
+      "grad_norm": 0.4342021644115448,
+      "learning_rate": 7.228358026812833e-05,
+      "loss": 0.5853999614715576,
+      "mean_token_accuracy": 0.82901301831007,
+      "num_tokens": 1076964.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060313630880579,
+      "eval_entropy": 0.6567474329069759,
+      "eval_loss": 0.6245927810668945,
+      "eval_mean_token_accuracy": 0.8211770050981072,
+      "eval_num_tokens": 1076964.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 440
+    },
+    {
+      "entropy": 0.639747454226017,
+      "epoch": 1.1085645355850422,
+      "grad_norm": 0.5338298082351685,
+      "learning_rate": 7.226619357333845e-05,
+      "loss": 0.5847910404205322,
+      "mean_token_accuracy": 0.8279331669211387,
+      "num_tokens": 1126064.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1085645355850422,
+      "eval_entropy": 0.6399250211340658,
+      "eval_loss": 0.6245712637901306,
+      "eval_mean_token_accuracy": 0.8209733875949731,
+      "eval_num_tokens": 1126064.0,
+      "eval_runtime": 98.6581,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 460
+    },
+    {
+      "entropy": 0.6248803421854973,
+      "epoch": 1.1568154402895054,
+      "grad_norm": 0.4402505159378052,
+      "learning_rate": 7.223858514173552e-05,
+      "loss": 0.5847790241241455,
+      "mean_token_accuracy": 0.8273933246731758,
+      "num_tokens": 1176912.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1568154402895054,
+      "eval_entropy": 0.6459465750147787,
+      "eval_loss": 0.615716278553009,
+      "eval_mean_token_accuracy": 0.8228987813665626,
+      "eval_num_tokens": 1176912.0,
+      "eval_runtime": 98.5987,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 480
+    },
+    {
+      "entropy": 0.6356995061039925,
+      "epoch": 1.2050663449939687,
+      "grad_norm": 0.5318694114685059,
+      "learning_rate": 7.220076278617914e-05,
+      "loss": 0.5902805328369141,
+      "mean_token_accuracy": 0.8279137581586837,
+      "num_tokens": 1223383.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2050663449939687,
+      "eval_entropy": 0.6509443488013879,
+      "eval_loss": 0.6174858808517456,
+      "eval_mean_token_accuracy": 0.8229377031996009,
+      "eval_num_tokens": 1223383.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 500
+    },
+    {
+      "entropy": 0.6378119193017483,
+      "epoch": 1.2533172496984317,
+      "grad_norm": 0.5385730266571045,
+      "learning_rate": 7.215273720994853e-05,
+      "loss": 0.5850183486938476,
+      "mean_token_accuracy": 0.8282771646976471,
+      "num_tokens": 1273534.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2533172496984317,
+      "eval_entropy": 0.6438969477508845,
+      "eval_loss": 0.6134508848190308,
+      "eval_mean_token_accuracy": 0.823984161186754,
+      "eval_num_tokens": 1273534.0,
+      "eval_runtime": 98.4814,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 520
+    },
+    {
+      "entropy": 0.6330174207687378,
+      "epoch": 1.301568154402895,
+      "grad_norm": 0.4950766861438751,
+      "learning_rate": 7.209452200371367e-05,
+      "loss": 0.5886627674102783,
+      "mean_token_accuracy": 0.828557687997818,
+      "num_tokens": 1323351.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.301568154402895,
+      "eval_entropy": 0.6342737025089478,
+      "eval_loss": 0.611145555973053,
+      "eval_mean_token_accuracy": 0.8250270244780551,
+      "eval_num_tokens": 1323351.0,
+      "eval_runtime": 98.6698,
+      "eval_samples_per_second": 14.391,
+      "eval_steps_per_second": 1.804,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270076125860214,
+      "epoch": 1.3498190591073582,
+      "grad_norm": 0.435604989528656,
+      "learning_rate": 7.20261336416893e-05,
+      "loss": 0.5834732532501221,
+      "mean_token_accuracy": 0.8290600389242172,
+      "num_tokens": 1371508.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3498190591073582,
+      "eval_entropy": 0.6624904408883513,
+      "eval_loss": 0.6063718795776367,
+      "eval_mean_token_accuracy": 0.825260305002834,
+      "eval_num_tokens": 1371508.0,
+      "eval_runtime": 98.6348,
+      "eval_samples_per_second": 14.397,
+      "eval_steps_per_second": 1.805,
+      "step": 560
+    },
+    {
+      "entropy": 0.6258987173438072,
+      "epoch": 1.3980699638118215,
+      "grad_norm": 0.45945441722869873,
+      "learning_rate": 7.194759147697293e-05,
+      "loss": 0.5705549716949463,
+      "mean_token_accuracy": 0.8329991281032563,
+      "num_tokens": 1415221.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3980699638118215,
+      "eval_entropy": 0.6500632069753797,
+      "eval_loss": 0.6059838533401489,
+      "eval_mean_token_accuracy": 0.8262855604793249,
+      "eval_num_tokens": 1415221.0,
+      "eval_runtime": 98.6469,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 580
+    },
+    {
+      "entropy": 0.6301489062607288,
+      "epoch": 1.4463208685162847,
+      "grad_norm": 0.5217350721359253,
+      "learning_rate": 7.18589177360681e-05,
+      "loss": 0.5778427124023438,
+      "mean_token_accuracy": 0.8305584967136384,
+      "num_tokens": 1463356.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4463208685162847,
+      "eval_entropy": 0.6478241879618569,
+      "eval_loss": 0.6014655828475952,
+      "eval_mean_token_accuracy": 0.8265940369514937,
+      "eval_num_tokens": 1463356.0,
+      "eval_runtime": 98.6523,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 600
+    },
+    {
+      "entropy": 0.6257719069719314,
+      "epoch": 1.494571773220748,
+      "grad_norm": 0.48310142755508423,
+      "learning_rate": 7.176013751259459e-05,
+      "loss": 0.5713310241699219,
+      "mean_token_accuracy": 0.8305533796548843,
+      "num_tokens": 1512340.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.494571773220748,
+      "eval_entropy": 0.6396401956510008,
+      "eval_loss": 0.5978795289993286,
+      "eval_mean_token_accuracy": 0.8273170630583603,
+      "eval_num_tokens": 1512340.0,
+      "eval_runtime": 98.4958,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 620
+    },
+    {
+      "entropy": 0.6162159442901611,
+      "epoch": 1.5428226779252112,
+      "grad_norm": 0.4500057101249695,
+      "learning_rate": 7.165127876018717e-05,
+      "loss": 0.565293264389038,
+      "mean_token_accuracy": 0.8311659932136536,
+      "num_tokens": 1560911.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5428226779252112,
+      "eval_entropy": 0.6381519051750054,
+      "eval_loss": 0.5963984131813049,
+      "eval_mean_token_accuracy": 0.8268409247478742,
+      "eval_num_tokens": 1560911.0,
+      "eval_runtime": 98.6803,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 640
+    },
+    {
+      "entropy": 0.6088747605681419,
+      "epoch": 1.5910735826296745,
+      "grad_norm": 0.46954667568206787,
+      "learning_rate": 7.153237228458511e-05,
+      "loss": 0.5602522850036621,
+      "mean_token_accuracy": 0.8345131903886795,
+      "num_tokens": 1609409.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5910735826296745,
+      "eval_entropy": 0.6413098875056492,
+      "eval_loss": 0.5938988327980042,
+      "eval_mean_token_accuracy": 0.828771137119679,
+      "eval_num_tokens": 1609409.0,
+      "eval_runtime": 98.4951,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 660
+    },
+    {
+      "entropy": 0.6209868296980858,
+      "epoch": 1.6393244873341375,
+      "grad_norm": 0.3867878317832947,
+      "learning_rate": 7.140345173491446e-05,
+      "loss": 0.5663797378540039,
+      "mean_token_accuracy": 0.8352224260568619,
+      "num_tokens": 1657430.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.6393244873341375,
+      "eval_entropy": 0.6182791748073664,
+      "eval_loss": 0.5919068455696106,
+      "eval_mean_token_accuracy": 0.8286421476455217,
+      "eval_num_tokens": 1657430.0,
+      "eval_runtime": 98.6211,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 680
+    },
+    {
+      "entropy": 0.6191923573613167,
+      "epoch": 1.6875753920386007,
+      "grad_norm": 0.40004226565361023,
+      "learning_rate": 7.12645535941658e-05,
+      "loss": 0.57517991065979,
+      "mean_token_accuracy": 0.8333286643028259,
+      "num_tokens": 1708058.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.6875753920386007,
+      "eval_entropy": 0.6244879199212856,
+      "eval_loss": 0.5897895693778992,
+      "eval_mean_token_accuracy": 0.8293621252092083,
+      "eval_num_tokens": 1708058.0,
+      "eval_runtime": 98.4974,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 700
+    },
+    {
+      "entropy": 0.6199282392859459,
+      "epoch": 1.7358262967430638,
+      "grad_norm": 0.36575791239738464,
+      "learning_rate": 7.111571716886997e-05,
+      "loss": 0.5717356681823731,
+      "mean_token_accuracy": 0.8321028783917427,
+      "num_tokens": 1761262.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.7358262967430638,
+      "eval_entropy": 0.6162440127201294,
+      "eval_loss": 0.5864423513412476,
+      "eval_mean_token_accuracy": 0.830874237786518,
+      "eval_num_tokens": 1761262.0,
+      "eval_runtime": 98.6109,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 720
+    },
+    {
+      "entropy": 0.6127455830574036,
+      "epoch": 1.784077201447527,
+      "grad_norm": 0.4778473973274231,
+      "learning_rate": 7.095698457797475e-05,
+      "loss": 0.5664590835571289,
+      "mean_token_accuracy": 0.8324130088090896,
+      "num_tokens": 1812127.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.784077201447527,
+      "eval_entropy": 0.6273092945640006,
+      "eval_loss": 0.5847376585006714,
+      "eval_mean_token_accuracy": 0.831051754817534,
+      "eval_num_tokens": 1812127.0,
+      "eval_runtime": 98.6513,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 740
+    },
+    {
+      "entropy": 0.6141144648194313,
+      "epoch": 1.8323281061519903,
+      "grad_norm": 0.6467089056968689,
+      "learning_rate": 7.078840074092574e-05,
+      "loss": 0.5548617362976074,
+      "mean_token_accuracy": 0.8342376738786698,
+      "num_tokens": 1858403.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.8323281061519903,
+      "eval_entropy": 0.5951976082967908,
+      "eval_loss": 0.5850171446800232,
+      "eval_mean_token_accuracy": 0.8303041870004675,
+      "eval_num_tokens": 1858403.0,
+      "eval_runtime": 98.6601,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 760
+    },
+    {
+      "entropy": 0.5961867064237595,
+      "epoch": 1.8805790108564535,
+      "grad_norm": 0.41280242800712585,
+      "learning_rate": 7.061001336495465e-05,
+      "loss": 0.5511267185211182,
+      "mean_token_accuracy": 0.8392465204000473,
+      "num_tokens": 1907361.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8805790108564535,
+      "eval_entropy": 0.6167017998320333,
+      "eval_loss": 0.5826791524887085,
+      "eval_mean_token_accuracy": 0.8309678050239434,
+      "eval_num_tokens": 1907361.0,
+      "eval_runtime": 98.6564,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 780
+    },
+    {
+      "entropy": 0.6025628983974457,
+      "epoch": 1.9288299155609168,
+      "grad_norm": 0.4078947901725769,
+      "learning_rate": 7.042187293157881e-05,
+      "loss": 0.5552026271820069,
+      "mean_token_accuracy": 0.8359778299927711,
+      "num_tokens": 1956605.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9288299155609168,
+      "eval_entropy": 0.6047008484601974,
+      "eval_loss": 0.5811123847961426,
+      "eval_mean_token_accuracy": 0.8320730804057603,
+      "eval_num_tokens": 1956605.0,
+      "eval_runtime": 98.6033,
+      "eval_samples_per_second": 14.401,
+      "eval_steps_per_second": 1.805,
+      "step": 800
+    },
+    {
+      "entropy": 0.6041995719075203,
+      "epoch": 1.97708082026538,
+      "grad_norm": 0.543890118598938,
+      "learning_rate": 7.022403268231535e-05,
+      "loss": 0.554384422302246,
+      "mean_token_accuracy": 0.8380729153752327,
+      "num_tokens": 2003308.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.97708082026538,
+      "eval_entropy": 0.6005127917179901,
+      "eval_loss": 0.5800131559371948,
+      "eval_mean_token_accuracy": 0.8313186547059691,
+      "eval_num_tokens": 2003308.0,
+      "eval_runtime": 98.4119,
+      "eval_samples_per_second": 14.429,
+      "eval_steps_per_second": 1.809,
+      "step": 820
+    },
+    {
+      "entropy": 0.567442889396961,
+      "epoch": 2.0241254523522314,
+      "grad_norm": 0.5015465021133423,
+      "learning_rate": 7.001654860361456e-05,
+      "loss": 0.5208251476287842,
+      "mean_token_accuracy": 0.8442784960453327,
+      "num_tokens": 2051331.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0241254523522314,
+      "eval_entropy": 0.5785037809878253,
+      "eval_loss": 0.5865591168403625,
+      "eval_mean_token_accuracy": 0.8307993857378371,
+      "eval_num_tokens": 2051331.0,
+      "eval_runtime": 98.5061,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 840
+    },
+    {
+      "entropy": 0.5552012033760547,
+      "epoch": 2.0723763570566947,
+      "grad_norm": 0.5975943207740784,
+      "learning_rate": 6.979947941101646e-05,
+      "loss": 0.5064489841461182,
+      "mean_token_accuracy": 0.8468396887183189,
+      "num_tokens": 2101528.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0723763570566947,
+      "eval_entropy": 0.5844205239515626,
+      "eval_loss": 0.585284948348999,
+      "eval_mean_token_accuracy": 0.8309797065311604,
+      "eval_num_tokens": 2101528.0,
+      "eval_runtime": 98.4982,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 860
+    },
+    {
+      "entropy": 0.5401638634502888,
+      "epoch": 2.120627261761158,
+      "grad_norm": 0.4194582998752594,
+      "learning_rate": 6.957288653253486e-05,
+      "loss": 0.49037814140319824,
+      "mean_token_accuracy": 0.8528496414422989,
+      "num_tokens": 2149550.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.120627261761158,
+      "eval_entropy": 0.574507936835289,
+      "eval_loss": 0.5836747288703918,
+      "eval_mean_token_accuracy": 0.8325241643391298,
+      "eval_num_tokens": 2149550.0,
+      "eval_runtime": 98.5592,
+      "eval_samples_per_second": 14.408,
+      "eval_steps_per_second": 1.806,
+      "step": 880
+    },
+    {
+      "entropy": 0.5606910541653634,
+      "epoch": 2.168878166465621,
+      "grad_norm": 0.5395355820655823,
+      "learning_rate": 6.933683409127401e-05,
+      "loss": 0.5089571475982666,
+      "mean_token_accuracy": 0.8459660410881042,
+      "num_tokens": 2202162.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.168878166465621,
+      "eval_entropy": 0.5779740023479033,
+      "eval_loss": 0.5816043615341187,
+      "eval_mean_token_accuracy": 0.8326318036304431,
+      "eval_num_tokens": 2202162.0,
+      "eval_runtime": 98.4386,
+      "eval_samples_per_second": 14.425,
+      "eval_steps_per_second": 1.808,
+      "step": 900
+    },
+    {
+      "entropy": 0.5380903735756875,
+      "epoch": 2.2171290711700844,
+      "grad_norm": 0.46588417887687683,
+      "learning_rate": 6.909138888728257e-05,
+      "loss": 0.4902662754058838,
+      "mean_token_accuracy": 0.8493925213813782,
+      "num_tokens": 2256539.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2171290711700844,
+      "eval_entropy": 0.570154261220707,
+      "eval_loss": 0.5827520489692688,
+      "eval_mean_token_accuracy": 0.8321914167216654,
+      "eval_num_tokens": 2256539.0,
+      "eval_runtime": 98.5674,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 920
+    },
+    {
+      "entropy": 0.558555257320404,
+      "epoch": 2.2653799758745476,
+      "grad_norm": 0.44576048851013184,
+      "learning_rate": 6.883662037864996e-05,
+      "loss": 0.5000699520111084,
+      "mean_token_accuracy": 0.8475695163011551,
+      "num_tokens": 2299698.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2653799758745476,
+      "eval_entropy": 0.5715417749761196,
+      "eval_loss": 0.5831062197685242,
+      "eval_mean_token_accuracy": 0.8327409899636601,
+      "eval_num_tokens": 2299698.0,
+      "eval_runtime": 98.568,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 940
+    },
+    {
+      "entropy": 0.5582522705197335,
+      "epoch": 2.313630880579011,
+      "grad_norm": 0.38966405391693115,
+      "learning_rate": 6.857260066185045e-05,
+      "loss": 0.5132341861724854,
+      "mean_token_accuracy": 0.8466864034533501,
+      "num_tokens": 2347134.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.313630880579011,
+      "eval_entropy": 0.5823918579334624,
+      "eval_loss": 0.5797588229179382,
+      "eval_mean_token_accuracy": 0.8326037566313583,
+      "eval_num_tokens": 2347134.0,
+      "eval_runtime": 98.6222,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 960
+    },
+    {
+      "entropy": 0.5557353265583516,
+      "epoch": 2.361881785283474,
+      "grad_norm": 0.42300811409950256,
+      "learning_rate": 6.829940445134089e-05,
+      "loss": 0.5035375118255615,
+      "mean_token_accuracy": 0.8479988545179367,
+      "num_tokens": 2394568.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.361881785283474,
+      "eval_entropy": 0.5542296482605881,
+      "eval_loss": 0.5800604820251465,
+      "eval_mean_token_accuracy": 0.8336404163515969,
+      "eval_num_tokens": 2394568.0,
+      "eval_runtime": 98.615,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 980
+    },
+    {
+      "entropy": 0.5492161035537719,
+      "epoch": 2.4101326899879374,
+      "grad_norm": 0.47482752799987793,
+      "learning_rate": 6.801710905841724e-05,
+      "loss": 0.5032052040100098,
+      "mean_token_accuracy": 0.8488596931099892,
+      "num_tokens": 2443253.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4101326899879374,
+      "eval_entropy": 0.5755258423558781,
+      "eval_loss": 0.5764774680137634,
+      "eval_mean_token_accuracy": 0.8336571763070781,
+      "eval_num_tokens": 2443253.0,
+      "eval_runtime": 98.4765,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5439704649150372,
+      "epoch": 2.4583835946924006,
+      "grad_norm": 0.3656543493270874,
+      "learning_rate": 6.772579436933644e-05,
+      "loss": 0.49903340339660646,
+      "mean_token_accuracy": 0.8499931409955025,
+      "num_tokens": 2495178.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4583835946924006,
+      "eval_entropy": 0.5712662409530597,
+      "eval_loss": 0.5777108073234558,
+      "eval_mean_token_accuracy": 0.8336941352051296,
+      "eval_num_tokens": 2495178.0,
+      "eval_runtime": 98.678,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 1020
+    },
+    {
+      "entropy": 0.5448883637785912,
+      "epoch": 2.5066344993968634,
+      "grad_norm": 0.3990268111228943,
+      "learning_rate": 6.742554282270959e-05,
+      "loss": 0.4990490436553955,
+      "mean_token_accuracy": 0.8488436549901962,
+      "num_tokens": 2541733.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5066344993968634,
+      "eval_entropy": 0.5816385483139017,
+      "eval_loss": 0.5749375820159912,
+      "eval_mean_token_accuracy": 0.833801462074344,
+      "eval_num_tokens": 2541733.0,
+      "eval_runtime": 98.6209,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 1040
+    },
+    {
+      "entropy": 0.5575303137302399,
+      "epoch": 2.554885404101327,
+      "grad_norm": 0.45015832781791687,
+      "learning_rate": 6.711643938617279e-05,
+      "loss": 0.5061063289642334,
+      "mean_token_accuracy": 0.8462719902396202,
+      "num_tokens": 2589479.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.554885404101327,
+      "eval_entropy": 0.5666238901655326,
+      "eval_loss": 0.574820876121521,
+      "eval_mean_token_accuracy": 0.8339746617869045,
+      "eval_num_tokens": 2589479.0,
+      "eval_runtime": 98.5332,
+      "eval_samples_per_second": 14.411,
+      "eval_steps_per_second": 1.806,
+      "step": 1060
+    },
+    {
+      "entropy": 0.5477629877626896,
+      "epoch": 2.60313630880579,
+      "grad_norm": 0.43583670258522034,
+      "learning_rate": 6.679857153234218e-05,
+      "loss": 0.4998013496398926,
+      "mean_token_accuracy": 0.8465577483177185,
+      "num_tokens": 2639134.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.60313630880579,
+      "eval_entropy": 0.5809191281206152,
+      "eval_loss": 0.5737497210502625,
+      "eval_mean_token_accuracy": 0.8338909467284599,
+      "eval_num_tokens": 2639134.0,
+      "eval_runtime": 98.5093,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 1080
+    },
+    {
+      "entropy": 0.5548589795827865,
+      "epoch": 2.651387213510253,
+      "grad_norm": 0.44145169854164124,
+      "learning_rate": 6.647202921406043e-05,
+      "loss": 0.5035099029541016,
+      "mean_token_accuracy": 0.8463503196835518,
+      "num_tokens": 2689324.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.651387213510253,
+      "eval_entropy": 0.5626686118292005,
+      "eval_loss": 0.5707846879959106,
+      "eval_mean_token_accuracy": 0.8349729531936432,
+      "eval_num_tokens": 2689324.0,
+      "eval_runtime": 98.473,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.57585367650304e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1120/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1120/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1120/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1120/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1120/trainer_state.json

@@ -0,0 +1,1210 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.6996381182147164,
+  "eval_steps": 20,
+  "global_step": 1120,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    },
+    {
+      "entropy": 0.8518766984343529,
+      "epoch": 0.28950542822677927,
+      "grad_norm": 0.635079026222229,
+      "learning_rate": 2.0729210702058663e-05,
+      "loss": 0.8058656692504883,
+      "mean_token_accuracy": 0.7847469851374627,
+      "num_tokens": 298041.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28950542822677927,
+      "eval_entropy": 0.838603059227547,
+      "eval_loss": 0.7992516756057739,
+      "eval_mean_token_accuracy": 0.7849764954508021,
+      "eval_num_tokens": 298041.0,
+      "eval_runtime": 98.5627,
+      "eval_samples_per_second": 14.407,
+      "eval_steps_per_second": 1.806,
+      "step": 120
+    },
+    {
+      "entropy": 0.8069365546107292,
+      "epoch": 0.33775633293124246,
+      "grad_norm": 0.5280472636222839,
+      "learning_rate": 2.421311166038785e-05,
+      "loss": 0.7562496662139893,
+      "mean_token_accuracy": 0.7897997036576271,
+      "num_tokens": 348955.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33775633293124246,
+      "eval_entropy": 0.8161682168419442,
+      "eval_loss": 0.7599642276763916,
+      "eval_mean_token_accuracy": 0.7912511644738444,
+      "eval_num_tokens": 348955.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 140
+    },
+    {
+      "entropy": 0.7952415809035301,
+      "epoch": 0.38600723763570566,
+      "grad_norm": 0.5387083888053894,
+      "learning_rate": 2.7697012618717034e-05,
+      "loss": 0.740808629989624,
+      "mean_token_accuracy": 0.7941806614398956,
+      "num_tokens": 395798.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38600723763570566,
+      "eval_entropy": 0.7856516191798649,
+      "eval_loss": 0.7324908971786499,
+      "eval_mean_token_accuracy": 0.7980317825011993,
+      "eval_num_tokens": 395798.0,
+      "eval_runtime": 98.6378,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 160
+    },
+    {
+      "entropy": 0.7537026956677437,
+      "epoch": 0.43425814234016885,
+      "grad_norm": 0.5521425604820251,
+      "learning_rate": 3.118091357704622e-05,
+      "loss": 0.7073616027832031,
+      "mean_token_accuracy": 0.8006261065602303,
+      "num_tokens": 446437.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43425814234016885,
+      "eval_entropy": 0.7470693678668375,
+      "eval_loss": 0.711143434047699,
+      "eval_mean_token_accuracy": 0.801459076029531,
+      "eval_num_tokens": 446437.0,
+      "eval_runtime": 98.6667,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 180
+    },
+    {
+      "entropy": 0.7328549668192863,
+      "epoch": 0.4825090470446321,
+      "grad_norm": 0.4901895821094513,
+      "learning_rate": 3.4664814535375406e-05,
+      "loss": 0.6874266624450683,
+      "mean_token_accuracy": 0.8066758885979652,
+      "num_tokens": 497682.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4825090470446321,
+      "eval_entropy": 0.7374835154983435,
+      "eval_loss": 0.6967977285385132,
+      "eval_mean_token_accuracy": 0.8055616520094068,
+      "eval_num_tokens": 497682.0,
+      "eval_runtime": 98.6234,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 200
+    },
+    {
+      "entropy": 0.7372006759047508,
+      "epoch": 0.5307599517490953,
+      "grad_norm": 0.5516473650932312,
+      "learning_rate": 3.814871549370459e-05,
+      "loss": 0.6906380653381348,
+      "mean_token_accuracy": 0.8057604253292083,
+      "num_tokens": 544670.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5307599517490953,
+      "eval_entropy": 0.710068748573239,
+      "eval_loss": 0.6860950589179993,
+      "eval_mean_token_accuracy": 0.8069355440943429,
+      "eval_num_tokens": 544670.0,
+      "eval_runtime": 98.4618,
+      "eval_samples_per_second": 14.422,
+      "eval_steps_per_second": 1.808,
+      "step": 220
+    },
+    {
+      "entropy": 0.7099859565496445,
+      "epoch": 0.5790108564535585,
+      "grad_norm": 0.457378625869751,
+      "learning_rate": 4.1632616452033785e-05,
+      "loss": 0.6674468517303467,
+      "mean_token_accuracy": 0.8087703332304954,
+      "num_tokens": 592926.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5790108564535585,
+      "eval_entropy": 0.717441116156203,
+      "eval_loss": 0.6765118837356567,
+      "eval_mean_token_accuracy": 0.8095786246690857,
+      "eval_num_tokens": 592926.0,
+      "eval_runtime": 98.6828,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 240
+    },
+    {
+      "entropy": 0.7170855522155761,
+      "epoch": 0.6272617611580217,
+      "grad_norm": 0.4604854881763458,
+      "learning_rate": 4.5116517410362974e-05,
+      "loss": 0.6647133827209473,
+      "mean_token_accuracy": 0.8115006521344185,
+      "num_tokens": 639077.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6272617611580217,
+      "eval_entropy": 0.7082702219486237,
+      "eval_loss": 0.6722468733787537,
+      "eval_mean_token_accuracy": 0.8100237367528208,
+      "eval_num_tokens": 639077.0,
+      "eval_runtime": 98.6242,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 260
+    },
+    {
+      "entropy": 0.691286925971508,
+      "epoch": 0.6755126658624849,
+      "grad_norm": 0.4626481831073761,
+      "learning_rate": 4.860041836869215e-05,
+      "loss": 0.6543298721313476,
+      "mean_token_accuracy": 0.8142751976847649,
+      "num_tokens": 691028.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6755126658624849,
+      "eval_entropy": 0.7341568747932992,
+      "eval_loss": 0.658997654914856,
+      "eval_mean_token_accuracy": 0.813439255684949,
+      "eval_num_tokens": 691028.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 280
+    },
+    {
+      "entropy": 0.691513468325138,
+      "epoch": 0.7237635705669482,
+      "grad_norm": 0.5434145331382751,
+      "learning_rate": 5.2084319327021346e-05,
+      "loss": 0.6393396377563476,
+      "mean_token_accuracy": 0.8172704800963402,
+      "num_tokens": 740255.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7237635705669482,
+      "eval_entropy": 0.699160492487168,
+      "eval_loss": 0.6522852182388306,
+      "eval_mean_token_accuracy": 0.8143363323774231,
+      "eval_num_tokens": 740255.0,
+      "eval_runtime": 98.6233,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 300
+    },
+    {
+      "entropy": 0.6918429866433143,
+      "epoch": 0.7720144752714113,
+      "grad_norm": 0.47131890058517456,
+      "learning_rate": 5.556822028535052e-05,
+      "loss": 0.6503581523895263,
+      "mean_token_accuracy": 0.8149650946259499,
+      "num_tokens": 789529.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7720144752714113,
+      "eval_entropy": 0.6949931701247611,
+      "eval_loss": 0.6464414596557617,
+      "eval_mean_token_accuracy": 0.816310279489903,
+      "eval_num_tokens": 789529.0,
+      "eval_runtime": 98.4759,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 320
+    },
+    {
+      "entropy": 0.6926439955830574,
+      "epoch": 0.8202653799758746,
+      "grad_norm": 0.46790942549705505,
+      "learning_rate": 5.905212124367972e-05,
+      "loss": 0.6453651428222656,
+      "mean_token_accuracy": 0.8171535924077034,
+      "num_tokens": 841584.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8202653799758746,
+      "eval_entropy": 0.6810062698433909,
+      "eval_loss": 0.6440736651420593,
+      "eval_mean_token_accuracy": 0.8166696479481258,
+      "eval_num_tokens": 841584.0,
+      "eval_runtime": 98.6479,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 340
+    },
+    {
+      "entropy": 0.7038845136761666,
+      "epoch": 0.8685162846803377,
+      "grad_norm": 0.5038695335388184,
+      "learning_rate": 6.253602220200891e-05,
+      "loss": 0.6558614253997803,
+      "mean_token_accuracy": 0.8127013072371483,
+      "num_tokens": 885769.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8685162846803377,
+      "eval_entropy": 0.6752923710292644,
+      "eval_loss": 0.6410524845123291,
+      "eval_mean_token_accuracy": 0.817534717950928,
+      "eval_num_tokens": 885769.0,
+      "eval_runtime": 98.5986,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 360
+    },
+    {
+      "entropy": 0.6897840186953544,
+      "epoch": 0.916767189384801,
+      "grad_norm": 0.5168097615242004,
+      "learning_rate": 6.601992316033808e-05,
+      "loss": 0.6328047752380371,
+      "mean_token_accuracy": 0.8181755512952804,
+      "num_tokens": 930056.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.916767189384801,
+      "eval_entropy": 0.6816390936964014,
+      "eval_loss": 0.6402509212493896,
+      "eval_mean_token_accuracy": 0.8172605586855599,
+      "eval_num_tokens": 930056.0,
+      "eval_runtime": 98.4787,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 380
+    },
+    {
+      "entropy": 0.6798740655183793,
+      "epoch": 0.9650180940892642,
+      "grad_norm": 0.4609147906303406,
+      "learning_rate": 6.950382411866727e-05,
+      "loss": 0.6358067989349365,
+      "mean_token_accuracy": 0.8193414479494094,
+      "num_tokens": 976892.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9650180940892642,
+      "eval_entropy": 0.6841439577970612,
+      "eval_loss": 0.6335230469703674,
+      "eval_mean_token_accuracy": 0.8190797965848045,
+      "eval_num_tokens": 976892.0,
+      "eval_runtime": 98.5983,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 400
+    },
+    {
+      "entropy": 0.6647876103719076,
+      "epoch": 1.0120627261761157,
+      "grad_norm": 0.46518462896347046,
+      "learning_rate": 7.229074030587624e-05,
+      "loss": 0.6112009525299072,
+      "mean_token_accuracy": 0.8212849283829714,
+      "num_tokens": 1024472.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0120627261761157,
+      "eval_entropy": 0.6612423688508151,
+      "eval_loss": 0.6311497688293457,
+      "eval_mean_token_accuracy": 0.8194578638237514,
+      "eval_num_tokens": 1024472.0,
+      "eval_runtime": 98.4872,
+      "eval_samples_per_second": 14.418,
+      "eval_steps_per_second": 1.807,
+      "step": 420
+    },
+    {
+      "entropy": 0.6288895070552826,
+      "epoch": 1.060313630880579,
+      "grad_norm": 0.4342021644115448,
+      "learning_rate": 7.228358026812833e-05,
+      "loss": 0.5853999614715576,
+      "mean_token_accuracy": 0.82901301831007,
+      "num_tokens": 1076964.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060313630880579,
+      "eval_entropy": 0.6567474329069759,
+      "eval_loss": 0.6245927810668945,
+      "eval_mean_token_accuracy": 0.8211770050981072,
+      "eval_num_tokens": 1076964.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 440
+    },
+    {
+      "entropy": 0.639747454226017,
+      "epoch": 1.1085645355850422,
+      "grad_norm": 0.5338298082351685,
+      "learning_rate": 7.226619357333845e-05,
+      "loss": 0.5847910404205322,
+      "mean_token_accuracy": 0.8279331669211387,
+      "num_tokens": 1126064.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1085645355850422,
+      "eval_entropy": 0.6399250211340658,
+      "eval_loss": 0.6245712637901306,
+      "eval_mean_token_accuracy": 0.8209733875949731,
+      "eval_num_tokens": 1126064.0,
+      "eval_runtime": 98.6581,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 460
+    },
+    {
+      "entropy": 0.6248803421854973,
+      "epoch": 1.1568154402895054,
+      "grad_norm": 0.4402505159378052,
+      "learning_rate": 7.223858514173552e-05,
+      "loss": 0.5847790241241455,
+      "mean_token_accuracy": 0.8273933246731758,
+      "num_tokens": 1176912.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1568154402895054,
+      "eval_entropy": 0.6459465750147787,
+      "eval_loss": 0.615716278553009,
+      "eval_mean_token_accuracy": 0.8228987813665626,
+      "eval_num_tokens": 1176912.0,
+      "eval_runtime": 98.5987,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 480
+    },
+    {
+      "entropy": 0.6356995061039925,
+      "epoch": 1.2050663449939687,
+      "grad_norm": 0.5318694114685059,
+      "learning_rate": 7.220076278617914e-05,
+      "loss": 0.5902805328369141,
+      "mean_token_accuracy": 0.8279137581586837,
+      "num_tokens": 1223383.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2050663449939687,
+      "eval_entropy": 0.6509443488013879,
+      "eval_loss": 0.6174858808517456,
+      "eval_mean_token_accuracy": 0.8229377031996009,
+      "eval_num_tokens": 1223383.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 500
+    },
+    {
+      "entropy": 0.6378119193017483,
+      "epoch": 1.2533172496984317,
+      "grad_norm": 0.5385730266571045,
+      "learning_rate": 7.215273720994853e-05,
+      "loss": 0.5850183486938476,
+      "mean_token_accuracy": 0.8282771646976471,
+      "num_tokens": 1273534.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2533172496984317,
+      "eval_entropy": 0.6438969477508845,
+      "eval_loss": 0.6134508848190308,
+      "eval_mean_token_accuracy": 0.823984161186754,
+      "eval_num_tokens": 1273534.0,
+      "eval_runtime": 98.4814,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 520
+    },
+    {
+      "entropy": 0.6330174207687378,
+      "epoch": 1.301568154402895,
+      "grad_norm": 0.4950766861438751,
+      "learning_rate": 7.209452200371367e-05,
+      "loss": 0.5886627674102783,
+      "mean_token_accuracy": 0.828557687997818,
+      "num_tokens": 1323351.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.301568154402895,
+      "eval_entropy": 0.6342737025089478,
+      "eval_loss": 0.611145555973053,
+      "eval_mean_token_accuracy": 0.8250270244780551,
+      "eval_num_tokens": 1323351.0,
+      "eval_runtime": 98.6698,
+      "eval_samples_per_second": 14.391,
+      "eval_steps_per_second": 1.804,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270076125860214,
+      "epoch": 1.3498190591073582,
+      "grad_norm": 0.435604989528656,
+      "learning_rate": 7.20261336416893e-05,
+      "loss": 0.5834732532501221,
+      "mean_token_accuracy": 0.8290600389242172,
+      "num_tokens": 1371508.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3498190591073582,
+      "eval_entropy": 0.6624904408883513,
+      "eval_loss": 0.6063718795776367,
+      "eval_mean_token_accuracy": 0.825260305002834,
+      "eval_num_tokens": 1371508.0,
+      "eval_runtime": 98.6348,
+      "eval_samples_per_second": 14.397,
+      "eval_steps_per_second": 1.805,
+      "step": 560
+    },
+    {
+      "entropy": 0.6258987173438072,
+      "epoch": 1.3980699638118215,
+      "grad_norm": 0.45945441722869873,
+      "learning_rate": 7.194759147697293e-05,
+      "loss": 0.5705549716949463,
+      "mean_token_accuracy": 0.8329991281032563,
+      "num_tokens": 1415221.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3980699638118215,
+      "eval_entropy": 0.6500632069753797,
+      "eval_loss": 0.6059838533401489,
+      "eval_mean_token_accuracy": 0.8262855604793249,
+      "eval_num_tokens": 1415221.0,
+      "eval_runtime": 98.6469,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 580
+    },
+    {
+      "entropy": 0.6301489062607288,
+      "epoch": 1.4463208685162847,
+      "grad_norm": 0.5217350721359253,
+      "learning_rate": 7.18589177360681e-05,
+      "loss": 0.5778427124023438,
+      "mean_token_accuracy": 0.8305584967136384,
+      "num_tokens": 1463356.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4463208685162847,
+      "eval_entropy": 0.6478241879618569,
+      "eval_loss": 0.6014655828475952,
+      "eval_mean_token_accuracy": 0.8265940369514937,
+      "eval_num_tokens": 1463356.0,
+      "eval_runtime": 98.6523,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 600
+    },
+    {
+      "entropy": 0.6257719069719314,
+      "epoch": 1.494571773220748,
+      "grad_norm": 0.48310142755508423,
+      "learning_rate": 7.176013751259459e-05,
+      "loss": 0.5713310241699219,
+      "mean_token_accuracy": 0.8305533796548843,
+      "num_tokens": 1512340.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.494571773220748,
+      "eval_entropy": 0.6396401956510008,
+      "eval_loss": 0.5978795289993286,
+      "eval_mean_token_accuracy": 0.8273170630583603,
+      "eval_num_tokens": 1512340.0,
+      "eval_runtime": 98.4958,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 620
+    },
+    {
+      "entropy": 0.6162159442901611,
+      "epoch": 1.5428226779252112,
+      "grad_norm": 0.4500057101249695,
+      "learning_rate": 7.165127876018717e-05,
+      "loss": 0.565293264389038,
+      "mean_token_accuracy": 0.8311659932136536,
+      "num_tokens": 1560911.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5428226779252112,
+      "eval_entropy": 0.6381519051750054,
+      "eval_loss": 0.5963984131813049,
+      "eval_mean_token_accuracy": 0.8268409247478742,
+      "eval_num_tokens": 1560911.0,
+      "eval_runtime": 98.6803,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 640
+    },
+    {
+      "entropy": 0.6088747605681419,
+      "epoch": 1.5910735826296745,
+      "grad_norm": 0.46954667568206787,
+      "learning_rate": 7.153237228458511e-05,
+      "loss": 0.5602522850036621,
+      "mean_token_accuracy": 0.8345131903886795,
+      "num_tokens": 1609409.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5910735826296745,
+      "eval_entropy": 0.6413098875056492,
+      "eval_loss": 0.5938988327980042,
+      "eval_mean_token_accuracy": 0.828771137119679,
+      "eval_num_tokens": 1609409.0,
+      "eval_runtime": 98.4951,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 660
+    },
+    {
+      "entropy": 0.6209868296980858,
+      "epoch": 1.6393244873341375,
+      "grad_norm": 0.3867878317832947,
+      "learning_rate": 7.140345173491446e-05,
+      "loss": 0.5663797378540039,
+      "mean_token_accuracy": 0.8352224260568619,
+      "num_tokens": 1657430.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.6393244873341375,
+      "eval_entropy": 0.6182791748073664,
+      "eval_loss": 0.5919068455696106,
+      "eval_mean_token_accuracy": 0.8286421476455217,
+      "eval_num_tokens": 1657430.0,
+      "eval_runtime": 98.6211,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 680
+    },
+    {
+      "entropy": 0.6191923573613167,
+      "epoch": 1.6875753920386007,
+      "grad_norm": 0.40004226565361023,
+      "learning_rate": 7.12645535941658e-05,
+      "loss": 0.57517991065979,
+      "mean_token_accuracy": 0.8333286643028259,
+      "num_tokens": 1708058.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.6875753920386007,
+      "eval_entropy": 0.6244879199212856,
+      "eval_loss": 0.5897895693778992,
+      "eval_mean_token_accuracy": 0.8293621252092083,
+      "eval_num_tokens": 1708058.0,
+      "eval_runtime": 98.4974,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 700
+    },
+    {
+      "entropy": 0.6199282392859459,
+      "epoch": 1.7358262967430638,
+      "grad_norm": 0.36575791239738464,
+      "learning_rate": 7.111571716886997e-05,
+      "loss": 0.5717356681823731,
+      "mean_token_accuracy": 0.8321028783917427,
+      "num_tokens": 1761262.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.7358262967430638,
+      "eval_entropy": 0.6162440127201294,
+      "eval_loss": 0.5864423513412476,
+      "eval_mean_token_accuracy": 0.830874237786518,
+      "eval_num_tokens": 1761262.0,
+      "eval_runtime": 98.6109,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 720
+    },
+    {
+      "entropy": 0.6127455830574036,
+      "epoch": 1.784077201447527,
+      "grad_norm": 0.4778473973274231,
+      "learning_rate": 7.095698457797475e-05,
+      "loss": 0.5664590835571289,
+      "mean_token_accuracy": 0.8324130088090896,
+      "num_tokens": 1812127.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.784077201447527,
+      "eval_entropy": 0.6273092945640006,
+      "eval_loss": 0.5847376585006714,
+      "eval_mean_token_accuracy": 0.831051754817534,
+      "eval_num_tokens": 1812127.0,
+      "eval_runtime": 98.6513,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 740
+    },
+    {
+      "entropy": 0.6141144648194313,
+      "epoch": 1.8323281061519903,
+      "grad_norm": 0.6467089056968689,
+      "learning_rate": 7.078840074092574e-05,
+      "loss": 0.5548617362976074,
+      "mean_token_accuracy": 0.8342376738786698,
+      "num_tokens": 1858403.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.8323281061519903,
+      "eval_entropy": 0.5951976082967908,
+      "eval_loss": 0.5850171446800232,
+      "eval_mean_token_accuracy": 0.8303041870004675,
+      "eval_num_tokens": 1858403.0,
+      "eval_runtime": 98.6601,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 760
+    },
+    {
+      "entropy": 0.5961867064237595,
+      "epoch": 1.8805790108564535,
+      "grad_norm": 0.41280242800712585,
+      "learning_rate": 7.061001336495465e-05,
+      "loss": 0.5511267185211182,
+      "mean_token_accuracy": 0.8392465204000473,
+      "num_tokens": 1907361.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8805790108564535,
+      "eval_entropy": 0.6167017998320333,
+      "eval_loss": 0.5826791524887085,
+      "eval_mean_token_accuracy": 0.8309678050239434,
+      "eval_num_tokens": 1907361.0,
+      "eval_runtime": 98.6564,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 780
+    },
+    {
+      "entropy": 0.6025628983974457,
+      "epoch": 1.9288299155609168,
+      "grad_norm": 0.4078947901725769,
+      "learning_rate": 7.042187293157881e-05,
+      "loss": 0.5552026271820069,
+      "mean_token_accuracy": 0.8359778299927711,
+      "num_tokens": 1956605.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9288299155609168,
+      "eval_entropy": 0.6047008484601974,
+      "eval_loss": 0.5811123847961426,
+      "eval_mean_token_accuracy": 0.8320730804057603,
+      "eval_num_tokens": 1956605.0,
+      "eval_runtime": 98.6033,
+      "eval_samples_per_second": 14.401,
+      "eval_steps_per_second": 1.805,
+      "step": 800
+    },
+    {
+      "entropy": 0.6041995719075203,
+      "epoch": 1.97708082026538,
+      "grad_norm": 0.543890118598938,
+      "learning_rate": 7.022403268231535e-05,
+      "loss": 0.554384422302246,
+      "mean_token_accuracy": 0.8380729153752327,
+      "num_tokens": 2003308.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.97708082026538,
+      "eval_entropy": 0.6005127917179901,
+      "eval_loss": 0.5800131559371948,
+      "eval_mean_token_accuracy": 0.8313186547059691,
+      "eval_num_tokens": 2003308.0,
+      "eval_runtime": 98.4119,
+      "eval_samples_per_second": 14.429,
+      "eval_steps_per_second": 1.809,
+      "step": 820
+    },
+    {
+      "entropy": 0.567442889396961,
+      "epoch": 2.0241254523522314,
+      "grad_norm": 0.5015465021133423,
+      "learning_rate": 7.001654860361456e-05,
+      "loss": 0.5208251476287842,
+      "mean_token_accuracy": 0.8442784960453327,
+      "num_tokens": 2051331.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0241254523522314,
+      "eval_entropy": 0.5785037809878253,
+      "eval_loss": 0.5865591168403625,
+      "eval_mean_token_accuracy": 0.8307993857378371,
+      "eval_num_tokens": 2051331.0,
+      "eval_runtime": 98.5061,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 840
+    },
+    {
+      "entropy": 0.5552012033760547,
+      "epoch": 2.0723763570566947,
+      "grad_norm": 0.5975943207740784,
+      "learning_rate": 6.979947941101646e-05,
+      "loss": 0.5064489841461182,
+      "mean_token_accuracy": 0.8468396887183189,
+      "num_tokens": 2101528.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0723763570566947,
+      "eval_entropy": 0.5844205239515626,
+      "eval_loss": 0.585284948348999,
+      "eval_mean_token_accuracy": 0.8309797065311604,
+      "eval_num_tokens": 2101528.0,
+      "eval_runtime": 98.4982,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 860
+    },
+    {
+      "entropy": 0.5401638634502888,
+      "epoch": 2.120627261761158,
+      "grad_norm": 0.4194582998752594,
+      "learning_rate": 6.957288653253486e-05,
+      "loss": 0.49037814140319824,
+      "mean_token_accuracy": 0.8528496414422989,
+      "num_tokens": 2149550.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.120627261761158,
+      "eval_entropy": 0.574507936835289,
+      "eval_loss": 0.5836747288703918,
+      "eval_mean_token_accuracy": 0.8325241643391298,
+      "eval_num_tokens": 2149550.0,
+      "eval_runtime": 98.5592,
+      "eval_samples_per_second": 14.408,
+      "eval_steps_per_second": 1.806,
+      "step": 880
+    },
+    {
+      "entropy": 0.5606910541653634,
+      "epoch": 2.168878166465621,
+      "grad_norm": 0.5395355820655823,
+      "learning_rate": 6.933683409127401e-05,
+      "loss": 0.5089571475982666,
+      "mean_token_accuracy": 0.8459660410881042,
+      "num_tokens": 2202162.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.168878166465621,
+      "eval_entropy": 0.5779740023479033,
+      "eval_loss": 0.5816043615341187,
+      "eval_mean_token_accuracy": 0.8326318036304431,
+      "eval_num_tokens": 2202162.0,
+      "eval_runtime": 98.4386,
+      "eval_samples_per_second": 14.425,
+      "eval_steps_per_second": 1.808,
+      "step": 900
+    },
+    {
+      "entropy": 0.5380903735756875,
+      "epoch": 2.2171290711700844,
+      "grad_norm": 0.46588417887687683,
+      "learning_rate": 6.909138888728257e-05,
+      "loss": 0.4902662754058838,
+      "mean_token_accuracy": 0.8493925213813782,
+      "num_tokens": 2256539.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2171290711700844,
+      "eval_entropy": 0.570154261220707,
+      "eval_loss": 0.5827520489692688,
+      "eval_mean_token_accuracy": 0.8321914167216654,
+      "eval_num_tokens": 2256539.0,
+      "eval_runtime": 98.5674,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 920
+    },
+    {
+      "entropy": 0.558555257320404,
+      "epoch": 2.2653799758745476,
+      "grad_norm": 0.44576048851013184,
+      "learning_rate": 6.883662037864996e-05,
+      "loss": 0.5000699520111084,
+      "mean_token_accuracy": 0.8475695163011551,
+      "num_tokens": 2299698.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2653799758745476,
+      "eval_entropy": 0.5715417749761196,
+      "eval_loss": 0.5831062197685242,
+      "eval_mean_token_accuracy": 0.8327409899636601,
+      "eval_num_tokens": 2299698.0,
+      "eval_runtime": 98.568,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 940
+    },
+    {
+      "entropy": 0.5582522705197335,
+      "epoch": 2.313630880579011,
+      "grad_norm": 0.38966405391693115,
+      "learning_rate": 6.857260066185045e-05,
+      "loss": 0.5132341861724854,
+      "mean_token_accuracy": 0.8466864034533501,
+      "num_tokens": 2347134.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.313630880579011,
+      "eval_entropy": 0.5823918579334624,
+      "eval_loss": 0.5797588229179382,
+      "eval_mean_token_accuracy": 0.8326037566313583,
+      "eval_num_tokens": 2347134.0,
+      "eval_runtime": 98.6222,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 960
+    },
+    {
+      "entropy": 0.5557353265583516,
+      "epoch": 2.361881785283474,
+      "grad_norm": 0.42300811409950256,
+      "learning_rate": 6.829940445134089e-05,
+      "loss": 0.5035375118255615,
+      "mean_token_accuracy": 0.8479988545179367,
+      "num_tokens": 2394568.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.361881785283474,
+      "eval_entropy": 0.5542296482605881,
+      "eval_loss": 0.5800604820251465,
+      "eval_mean_token_accuracy": 0.8336404163515969,
+      "eval_num_tokens": 2394568.0,
+      "eval_runtime": 98.615,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 980
+    },
+    {
+      "entropy": 0.5492161035537719,
+      "epoch": 2.4101326899879374,
+      "grad_norm": 0.47482752799987793,
+      "learning_rate": 6.801710905841724e-05,
+      "loss": 0.5032052040100098,
+      "mean_token_accuracy": 0.8488596931099892,
+      "num_tokens": 2443253.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4101326899879374,
+      "eval_entropy": 0.5755258423558781,
+      "eval_loss": 0.5764774680137634,
+      "eval_mean_token_accuracy": 0.8336571763070781,
+      "eval_num_tokens": 2443253.0,
+      "eval_runtime": 98.4765,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5439704649150372,
+      "epoch": 2.4583835946924006,
+      "grad_norm": 0.3656543493270874,
+      "learning_rate": 6.772579436933644e-05,
+      "loss": 0.49903340339660646,
+      "mean_token_accuracy": 0.8499931409955025,
+      "num_tokens": 2495178.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4583835946924006,
+      "eval_entropy": 0.5712662409530597,
+      "eval_loss": 0.5777108073234558,
+      "eval_mean_token_accuracy": 0.8336941352051296,
+      "eval_num_tokens": 2495178.0,
+      "eval_runtime": 98.678,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 1020
+    },
+    {
+      "entropy": 0.5448883637785912,
+      "epoch": 2.5066344993968634,
+      "grad_norm": 0.3990268111228943,
+      "learning_rate": 6.742554282270959e-05,
+      "loss": 0.4990490436553955,
+      "mean_token_accuracy": 0.8488436549901962,
+      "num_tokens": 2541733.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5066344993968634,
+      "eval_entropy": 0.5816385483139017,
+      "eval_loss": 0.5749375820159912,
+      "eval_mean_token_accuracy": 0.833801462074344,
+      "eval_num_tokens": 2541733.0,
+      "eval_runtime": 98.6209,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 1040
+    },
+    {
+      "entropy": 0.5575303137302399,
+      "epoch": 2.554885404101327,
+      "grad_norm": 0.45015832781791687,
+      "learning_rate": 6.711643938617279e-05,
+      "loss": 0.5061063289642334,
+      "mean_token_accuracy": 0.8462719902396202,
+      "num_tokens": 2589479.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.554885404101327,
+      "eval_entropy": 0.5666238901655326,
+      "eval_loss": 0.574820876121521,
+      "eval_mean_token_accuracy": 0.8339746617869045,
+      "eval_num_tokens": 2589479.0,
+      "eval_runtime": 98.5332,
+      "eval_samples_per_second": 14.411,
+      "eval_steps_per_second": 1.806,
+      "step": 1060
+    },
+    {
+      "entropy": 0.5477629877626896,
+      "epoch": 2.60313630880579,
+      "grad_norm": 0.43583670258522034,
+      "learning_rate": 6.679857153234218e-05,
+      "loss": 0.4998013496398926,
+      "mean_token_accuracy": 0.8465577483177185,
+      "num_tokens": 2639134.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.60313630880579,
+      "eval_entropy": 0.5809191281206152,
+      "eval_loss": 0.5737497210502625,
+      "eval_mean_token_accuracy": 0.8338909467284599,
+      "eval_num_tokens": 2639134.0,
+      "eval_runtime": 98.5093,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 1080
+    },
+    {
+      "entropy": 0.5548589795827865,
+      "epoch": 2.651387213510253,
+      "grad_norm": 0.44145169854164124,
+      "learning_rate": 6.647202921406043e-05,
+      "loss": 0.5035099029541016,
+      "mean_token_accuracy": 0.8463503196835518,
+      "num_tokens": 2689324.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.651387213510253,
+      "eval_entropy": 0.5626686118292005,
+      "eval_loss": 0.5707846879959106,
+      "eval_mean_token_accuracy": 0.8349729531936432,
+      "eval_num_tokens": 2689324.0,
+      "eval_runtime": 98.473,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1100
+    },
+    {
+      "entropy": 0.5568635657429695,
+      "epoch": 2.6996381182147164,
+      "grad_norm": 0.38482189178466797,
+      "learning_rate": 6.613690483894096e-05,
+      "loss": 0.5090720176696777,
+      "mean_token_accuracy": 0.8481037884950637,
+      "num_tokens": 2736182.0,
+      "step": 1120
+    },
+    {
+      "epoch": 2.6996381182147164,
+      "eval_entropy": 0.5752792709998871,
+      "eval_loss": 0.5706034898757935,
+      "eval_mean_token_accuracy": 0.8350465498613507,
+      "eval_num_tokens": 2736182.0,
+      "eval_runtime": 98.6524,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 1120
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.68034904428544e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1140/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1140/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1140/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1140/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1140/trainer_state.json

@@ -0,0 +1,1231 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.7478890229191797,
+  "eval_steps": 20,
+  "global_step": 1140,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.8880569130182265,
+      "epoch": 0.04825090470446321,
+      "grad_norm": 0.5548511743545532,
+      "learning_rate": 3.3097059104127273e-06,
+      "loss": 1.863978958129883,
+      "mean_token_accuracy": 0.621908786892891,
+      "num_tokens": 50237.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.04825090470446321,
+      "eval_entropy": 1.8981187470843284,
+      "eval_loss": 1.9387791156768799,
+      "eval_mean_token_accuracy": 0.6191755044326354,
+      "eval_num_tokens": 50237.0,
+      "eval_runtime": 99.1278,
+      "eval_samples_per_second": 14.325,
+      "eval_steps_per_second": 1.796,
+      "step": 20
+    },
+    {
+      "entropy": 1.845469006896019,
+      "epoch": 0.09650180940892641,
+      "grad_norm": 0.3859543204307556,
+      "learning_rate": 6.793606868741914e-06,
+      "loss": 1.8429210662841797,
+      "mean_token_accuracy": 0.6286612510681152,
+      "num_tokens": 101098.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.09650180940892641,
+      "eval_entropy": 1.8266146584842982,
+      "eval_loss": 1.8124289512634277,
+      "eval_mean_token_accuracy": 0.636085176400924,
+      "eval_num_tokens": 101098.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 40
+    },
+    {
+      "entropy": 1.6761731654405594,
+      "epoch": 0.14475271411338964,
+      "grad_norm": 0.763229489326477,
+      "learning_rate": 1.0277507827071102e-05,
+      "loss": 1.59182710647583,
+      "mean_token_accuracy": 0.6615026861429214,
+      "num_tokens": 150385.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.14475271411338964,
+      "eval_entropy": 1.448009472884489,
+      "eval_loss": 1.4045010805130005,
+      "eval_mean_token_accuracy": 0.6956760652279586,
+      "eval_num_tokens": 150385.0,
+      "eval_runtime": 98.6634,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 60
+    },
+    {
+      "entropy": 1.2918378412723541,
+      "epoch": 0.19300361881785283,
+      "grad_norm": 0.6131361126899719,
+      "learning_rate": 1.3761408785400288e-05,
+      "loss": 1.2077620506286622,
+      "mean_token_accuracy": 0.715845774114132,
+      "num_tokens": 197354.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.19300361881785283,
+      "eval_entropy": 1.0959123356288738,
+      "eval_loss": 1.025644063949585,
+      "eval_mean_token_accuracy": 0.7453202035989654,
+      "eval_num_tokens": 197354.0,
+      "eval_runtime": 98.6386,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 80
+    },
+    {
+      "entropy": 0.9881208389997482,
+      "epoch": 0.24125452352231605,
+      "grad_norm": 0.4983534514904022,
+      "learning_rate": 1.7245309743729473e-05,
+      "loss": 0.9109245300292969,
+      "mean_token_accuracy": 0.7622973442077636,
+      "num_tokens": 248557.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.24125452352231605,
+      "eval_entropy": 0.9157560136211053,
+      "eval_loss": 0.8661546111106873,
+      "eval_mean_token_accuracy": 0.7745042727904373,
+      "eval_num_tokens": 248557.0,
+      "eval_runtime": 98.5282,
+      "eval_samples_per_second": 14.412,
+      "eval_steps_per_second": 1.807,
+      "step": 100
+    },
+    {
+      "entropy": 0.8518766984343529,
+      "epoch": 0.28950542822677927,
+      "grad_norm": 0.635079026222229,
+      "learning_rate": 2.0729210702058663e-05,
+      "loss": 0.8058656692504883,
+      "mean_token_accuracy": 0.7847469851374627,
+      "num_tokens": 298041.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.28950542822677927,
+      "eval_entropy": 0.838603059227547,
+      "eval_loss": 0.7992516756057739,
+      "eval_mean_token_accuracy": 0.7849764954508021,
+      "eval_num_tokens": 298041.0,
+      "eval_runtime": 98.5627,
+      "eval_samples_per_second": 14.407,
+      "eval_steps_per_second": 1.806,
+      "step": 120
+    },
+    {
+      "entropy": 0.8069365546107292,
+      "epoch": 0.33775633293124246,
+      "grad_norm": 0.5280472636222839,
+      "learning_rate": 2.421311166038785e-05,
+      "loss": 0.7562496662139893,
+      "mean_token_accuracy": 0.7897997036576271,
+      "num_tokens": 348955.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.33775633293124246,
+      "eval_entropy": 0.8161682168419442,
+      "eval_loss": 0.7599642276763916,
+      "eval_mean_token_accuracy": 0.7912511644738444,
+      "eval_num_tokens": 348955.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 140
+    },
+    {
+      "entropy": 0.7952415809035301,
+      "epoch": 0.38600723763570566,
+      "grad_norm": 0.5387083888053894,
+      "learning_rate": 2.7697012618717034e-05,
+      "loss": 0.740808629989624,
+      "mean_token_accuracy": 0.7941806614398956,
+      "num_tokens": 395798.0,
+      "step": 160
+    },
+    {
+      "epoch": 0.38600723763570566,
+      "eval_entropy": 0.7856516191798649,
+      "eval_loss": 0.7324908971786499,
+      "eval_mean_token_accuracy": 0.7980317825011993,
+      "eval_num_tokens": 395798.0,
+      "eval_runtime": 98.6378,
+      "eval_samples_per_second": 14.396,
+      "eval_steps_per_second": 1.805,
+      "step": 160
+    },
+    {
+      "entropy": 0.7537026956677437,
+      "epoch": 0.43425814234016885,
+      "grad_norm": 0.5521425604820251,
+      "learning_rate": 3.118091357704622e-05,
+      "loss": 0.7073616027832031,
+      "mean_token_accuracy": 0.8006261065602303,
+      "num_tokens": 446437.0,
+      "step": 180
+    },
+    {
+      "epoch": 0.43425814234016885,
+      "eval_entropy": 0.7470693678668375,
+      "eval_loss": 0.711143434047699,
+      "eval_mean_token_accuracy": 0.801459076029531,
+      "eval_num_tokens": 446437.0,
+      "eval_runtime": 98.6667,
+      "eval_samples_per_second": 14.392,
+      "eval_steps_per_second": 1.804,
+      "step": 180
+    },
+    {
+      "entropy": 0.7328549668192863,
+      "epoch": 0.4825090470446321,
+      "grad_norm": 0.4901895821094513,
+      "learning_rate": 3.4664814535375406e-05,
+      "loss": 0.6874266624450683,
+      "mean_token_accuracy": 0.8066758885979652,
+      "num_tokens": 497682.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.4825090470446321,
+      "eval_entropy": 0.7374835154983435,
+      "eval_loss": 0.6967977285385132,
+      "eval_mean_token_accuracy": 0.8055616520094068,
+      "eval_num_tokens": 497682.0,
+      "eval_runtime": 98.6234,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 200
+    },
+    {
+      "entropy": 0.7372006759047508,
+      "epoch": 0.5307599517490953,
+      "grad_norm": 0.5516473650932312,
+      "learning_rate": 3.814871549370459e-05,
+      "loss": 0.6906380653381348,
+      "mean_token_accuracy": 0.8057604253292083,
+      "num_tokens": 544670.0,
+      "step": 220
+    },
+    {
+      "epoch": 0.5307599517490953,
+      "eval_entropy": 0.710068748573239,
+      "eval_loss": 0.6860950589179993,
+      "eval_mean_token_accuracy": 0.8069355440943429,
+      "eval_num_tokens": 544670.0,
+      "eval_runtime": 98.4618,
+      "eval_samples_per_second": 14.422,
+      "eval_steps_per_second": 1.808,
+      "step": 220
+    },
+    {
+      "entropy": 0.7099859565496445,
+      "epoch": 0.5790108564535585,
+      "grad_norm": 0.457378625869751,
+      "learning_rate": 4.1632616452033785e-05,
+      "loss": 0.6674468517303467,
+      "mean_token_accuracy": 0.8087703332304954,
+      "num_tokens": 592926.0,
+      "step": 240
+    },
+    {
+      "epoch": 0.5790108564535585,
+      "eval_entropy": 0.717441116156203,
+      "eval_loss": 0.6765118837356567,
+      "eval_mean_token_accuracy": 0.8095786246690857,
+      "eval_num_tokens": 592926.0,
+      "eval_runtime": 98.6828,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 240
+    },
+    {
+      "entropy": 0.7170855522155761,
+      "epoch": 0.6272617611580217,
+      "grad_norm": 0.4604854881763458,
+      "learning_rate": 4.5116517410362974e-05,
+      "loss": 0.6647133827209473,
+      "mean_token_accuracy": 0.8115006521344185,
+      "num_tokens": 639077.0,
+      "step": 260
+    },
+    {
+      "epoch": 0.6272617611580217,
+      "eval_entropy": 0.7082702219486237,
+      "eval_loss": 0.6722468733787537,
+      "eval_mean_token_accuracy": 0.8100237367528208,
+      "eval_num_tokens": 639077.0,
+      "eval_runtime": 98.6242,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 260
+    },
+    {
+      "entropy": 0.691286925971508,
+      "epoch": 0.6755126658624849,
+      "grad_norm": 0.4626481831073761,
+      "learning_rate": 4.860041836869215e-05,
+      "loss": 0.6543298721313476,
+      "mean_token_accuracy": 0.8142751976847649,
+      "num_tokens": 691028.0,
+      "step": 280
+    },
+    {
+      "epoch": 0.6755126658624849,
+      "eval_entropy": 0.7341568747932992,
+      "eval_loss": 0.658997654914856,
+      "eval_mean_token_accuracy": 0.813439255684949,
+      "eval_num_tokens": 691028.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 280
+    },
+    {
+      "entropy": 0.691513468325138,
+      "epoch": 0.7237635705669482,
+      "grad_norm": 0.5434145331382751,
+      "learning_rate": 5.2084319327021346e-05,
+      "loss": 0.6393396377563476,
+      "mean_token_accuracy": 0.8172704800963402,
+      "num_tokens": 740255.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7237635705669482,
+      "eval_entropy": 0.699160492487168,
+      "eval_loss": 0.6522852182388306,
+      "eval_mean_token_accuracy": 0.8143363323774231,
+      "eval_num_tokens": 740255.0,
+      "eval_runtime": 98.6233,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 300
+    },
+    {
+      "entropy": 0.6918429866433143,
+      "epoch": 0.7720144752714113,
+      "grad_norm": 0.47131890058517456,
+      "learning_rate": 5.556822028535052e-05,
+      "loss": 0.6503581523895263,
+      "mean_token_accuracy": 0.8149650946259499,
+      "num_tokens": 789529.0,
+      "step": 320
+    },
+    {
+      "epoch": 0.7720144752714113,
+      "eval_entropy": 0.6949931701247611,
+      "eval_loss": 0.6464414596557617,
+      "eval_mean_token_accuracy": 0.816310279489903,
+      "eval_num_tokens": 789529.0,
+      "eval_runtime": 98.4759,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 320
+    },
+    {
+      "entropy": 0.6926439955830574,
+      "epoch": 0.8202653799758746,
+      "grad_norm": 0.46790942549705505,
+      "learning_rate": 5.905212124367972e-05,
+      "loss": 0.6453651428222656,
+      "mean_token_accuracy": 0.8171535924077034,
+      "num_tokens": 841584.0,
+      "step": 340
+    },
+    {
+      "epoch": 0.8202653799758746,
+      "eval_entropy": 0.6810062698433909,
+      "eval_loss": 0.6440736651420593,
+      "eval_mean_token_accuracy": 0.8166696479481258,
+      "eval_num_tokens": 841584.0,
+      "eval_runtime": 98.6479,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 340
+    },
+    {
+      "entropy": 0.7038845136761666,
+      "epoch": 0.8685162846803377,
+      "grad_norm": 0.5038695335388184,
+      "learning_rate": 6.253602220200891e-05,
+      "loss": 0.6558614253997803,
+      "mean_token_accuracy": 0.8127013072371483,
+      "num_tokens": 885769.0,
+      "step": 360
+    },
+    {
+      "epoch": 0.8685162846803377,
+      "eval_entropy": 0.6752923710292644,
+      "eval_loss": 0.6410524845123291,
+      "eval_mean_token_accuracy": 0.817534717950928,
+      "eval_num_tokens": 885769.0,
+      "eval_runtime": 98.5986,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 360
+    },
+    {
+      "entropy": 0.6897840186953544,
+      "epoch": 0.916767189384801,
+      "grad_norm": 0.5168097615242004,
+      "learning_rate": 6.601992316033808e-05,
+      "loss": 0.6328047752380371,
+      "mean_token_accuracy": 0.8181755512952804,
+      "num_tokens": 930056.0,
+      "step": 380
+    },
+    {
+      "epoch": 0.916767189384801,
+      "eval_entropy": 0.6816390936964014,
+      "eval_loss": 0.6402509212493896,
+      "eval_mean_token_accuracy": 0.8172605586855599,
+      "eval_num_tokens": 930056.0,
+      "eval_runtime": 98.4787,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 380
+    },
+    {
+      "entropy": 0.6798740655183793,
+      "epoch": 0.9650180940892642,
+      "grad_norm": 0.4609147906303406,
+      "learning_rate": 6.950382411866727e-05,
+      "loss": 0.6358067989349365,
+      "mean_token_accuracy": 0.8193414479494094,
+      "num_tokens": 976892.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.9650180940892642,
+      "eval_entropy": 0.6841439577970612,
+      "eval_loss": 0.6335230469703674,
+      "eval_mean_token_accuracy": 0.8190797965848045,
+      "eval_num_tokens": 976892.0,
+      "eval_runtime": 98.5983,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 400
+    },
+    {
+      "entropy": 0.6647876103719076,
+      "epoch": 1.0120627261761157,
+      "grad_norm": 0.46518462896347046,
+      "learning_rate": 7.229074030587624e-05,
+      "loss": 0.6112009525299072,
+      "mean_token_accuracy": 0.8212849283829714,
+      "num_tokens": 1024472.0,
+      "step": 420
+    },
+    {
+      "epoch": 1.0120627261761157,
+      "eval_entropy": 0.6612423688508151,
+      "eval_loss": 0.6311497688293457,
+      "eval_mean_token_accuracy": 0.8194578638237514,
+      "eval_num_tokens": 1024472.0,
+      "eval_runtime": 98.4872,
+      "eval_samples_per_second": 14.418,
+      "eval_steps_per_second": 1.807,
+      "step": 420
+    },
+    {
+      "entropy": 0.6288895070552826,
+      "epoch": 1.060313630880579,
+      "grad_norm": 0.4342021644115448,
+      "learning_rate": 7.228358026812833e-05,
+      "loss": 0.5853999614715576,
+      "mean_token_accuracy": 0.82901301831007,
+      "num_tokens": 1076964.0,
+      "step": 440
+    },
+    {
+      "epoch": 1.060313630880579,
+      "eval_entropy": 0.6567474329069759,
+      "eval_loss": 0.6245927810668945,
+      "eval_mean_token_accuracy": 0.8211770050981072,
+      "eval_num_tokens": 1076964.0,
+      "eval_runtime": 98.6093,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 440
+    },
+    {
+      "entropy": 0.639747454226017,
+      "epoch": 1.1085645355850422,
+      "grad_norm": 0.5338298082351685,
+      "learning_rate": 7.226619357333845e-05,
+      "loss": 0.5847910404205322,
+      "mean_token_accuracy": 0.8279331669211387,
+      "num_tokens": 1126064.0,
+      "step": 460
+    },
+    {
+      "epoch": 1.1085645355850422,
+      "eval_entropy": 0.6399250211340658,
+      "eval_loss": 0.6245712637901306,
+      "eval_mean_token_accuracy": 0.8209733875949731,
+      "eval_num_tokens": 1126064.0,
+      "eval_runtime": 98.6581,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 460
+    },
+    {
+      "entropy": 0.6248803421854973,
+      "epoch": 1.1568154402895054,
+      "grad_norm": 0.4402505159378052,
+      "learning_rate": 7.223858514173552e-05,
+      "loss": 0.5847790241241455,
+      "mean_token_accuracy": 0.8273933246731758,
+      "num_tokens": 1176912.0,
+      "step": 480
+    },
+    {
+      "epoch": 1.1568154402895054,
+      "eval_entropy": 0.6459465750147787,
+      "eval_loss": 0.615716278553009,
+      "eval_mean_token_accuracy": 0.8228987813665626,
+      "eval_num_tokens": 1176912.0,
+      "eval_runtime": 98.5987,
+      "eval_samples_per_second": 14.402,
+      "eval_steps_per_second": 1.805,
+      "step": 480
+    },
+    {
+      "entropy": 0.6356995061039925,
+      "epoch": 1.2050663449939687,
+      "grad_norm": 0.5318694114685059,
+      "learning_rate": 7.220076278617914e-05,
+      "loss": 0.5902805328369141,
+      "mean_token_accuracy": 0.8279137581586837,
+      "num_tokens": 1223383.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2050663449939687,
+      "eval_entropy": 0.6509443488013879,
+      "eval_loss": 0.6174858808517456,
+      "eval_mean_token_accuracy": 0.8229377031996009,
+      "eval_num_tokens": 1223383.0,
+      "eval_runtime": 98.6156,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 500
+    },
+    {
+      "entropy": 0.6378119193017483,
+      "epoch": 1.2533172496984317,
+      "grad_norm": 0.5385730266571045,
+      "learning_rate": 7.215273720994853e-05,
+      "loss": 0.5850183486938476,
+      "mean_token_accuracy": 0.8282771646976471,
+      "num_tokens": 1273534.0,
+      "step": 520
+    },
+    {
+      "epoch": 1.2533172496984317,
+      "eval_entropy": 0.6438969477508845,
+      "eval_loss": 0.6134508848190308,
+      "eval_mean_token_accuracy": 0.823984161186754,
+      "eval_num_tokens": 1273534.0,
+      "eval_runtime": 98.4814,
+      "eval_samples_per_second": 14.419,
+      "eval_steps_per_second": 1.807,
+      "step": 520
+    },
+    {
+      "entropy": 0.6330174207687378,
+      "epoch": 1.301568154402895,
+      "grad_norm": 0.4950766861438751,
+      "learning_rate": 7.209452200371367e-05,
+      "loss": 0.5886627674102783,
+      "mean_token_accuracy": 0.828557687997818,
+      "num_tokens": 1323351.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.301568154402895,
+      "eval_entropy": 0.6342737025089478,
+      "eval_loss": 0.611145555973053,
+      "eval_mean_token_accuracy": 0.8250270244780551,
+      "eval_num_tokens": 1323351.0,
+      "eval_runtime": 98.6698,
+      "eval_samples_per_second": 14.391,
+      "eval_steps_per_second": 1.804,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270076125860214,
+      "epoch": 1.3498190591073582,
+      "grad_norm": 0.435604989528656,
+      "learning_rate": 7.20261336416893e-05,
+      "loss": 0.5834732532501221,
+      "mean_token_accuracy": 0.8290600389242172,
+      "num_tokens": 1371508.0,
+      "step": 560
+    },
+    {
+      "epoch": 1.3498190591073582,
+      "eval_entropy": 0.6624904408883513,
+      "eval_loss": 0.6063718795776367,
+      "eval_mean_token_accuracy": 0.825260305002834,
+      "eval_num_tokens": 1371508.0,
+      "eval_runtime": 98.6348,
+      "eval_samples_per_second": 14.397,
+      "eval_steps_per_second": 1.805,
+      "step": 560
+    },
+    {
+      "entropy": 0.6258987173438072,
+      "epoch": 1.3980699638118215,
+      "grad_norm": 0.45945441722869873,
+      "learning_rate": 7.194759147697293e-05,
+      "loss": 0.5705549716949463,
+      "mean_token_accuracy": 0.8329991281032563,
+      "num_tokens": 1415221.0,
+      "step": 580
+    },
+    {
+      "epoch": 1.3980699638118215,
+      "eval_entropy": 0.6500632069753797,
+      "eval_loss": 0.6059838533401489,
+      "eval_mean_token_accuracy": 0.8262855604793249,
+      "eval_num_tokens": 1415221.0,
+      "eval_runtime": 98.6469,
+      "eval_samples_per_second": 14.395,
+      "eval_steps_per_second": 1.804,
+      "step": 580
+    },
+    {
+      "entropy": 0.6301489062607288,
+      "epoch": 1.4463208685162847,
+      "grad_norm": 0.5217350721359253,
+      "learning_rate": 7.18589177360681e-05,
+      "loss": 0.5778427124023438,
+      "mean_token_accuracy": 0.8305584967136384,
+      "num_tokens": 1463356.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.4463208685162847,
+      "eval_entropy": 0.6478241879618569,
+      "eval_loss": 0.6014655828475952,
+      "eval_mean_token_accuracy": 0.8265940369514937,
+      "eval_num_tokens": 1463356.0,
+      "eval_runtime": 98.6523,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 600
+    },
+    {
+      "entropy": 0.6257719069719314,
+      "epoch": 1.494571773220748,
+      "grad_norm": 0.48310142755508423,
+      "learning_rate": 7.176013751259459e-05,
+      "loss": 0.5713310241699219,
+      "mean_token_accuracy": 0.8305533796548843,
+      "num_tokens": 1512340.0,
+      "step": 620
+    },
+    {
+      "epoch": 1.494571773220748,
+      "eval_entropy": 0.6396401956510008,
+      "eval_loss": 0.5978795289993286,
+      "eval_mean_token_accuracy": 0.8273170630583603,
+      "eval_num_tokens": 1512340.0,
+      "eval_runtime": 98.4958,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 620
+    },
+    {
+      "entropy": 0.6162159442901611,
+      "epoch": 1.5428226779252112,
+      "grad_norm": 0.4500057101249695,
+      "learning_rate": 7.165127876018717e-05,
+      "loss": 0.565293264389038,
+      "mean_token_accuracy": 0.8311659932136536,
+      "num_tokens": 1560911.0,
+      "step": 640
+    },
+    {
+      "epoch": 1.5428226779252112,
+      "eval_entropy": 0.6381519051750054,
+      "eval_loss": 0.5963984131813049,
+      "eval_mean_token_accuracy": 0.8268409247478742,
+      "eval_num_tokens": 1560911.0,
+      "eval_runtime": 98.6803,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 640
+    },
+    {
+      "entropy": 0.6088747605681419,
+      "epoch": 1.5910735826296745,
+      "grad_norm": 0.46954667568206787,
+      "learning_rate": 7.153237228458511e-05,
+      "loss": 0.5602522850036621,
+      "mean_token_accuracy": 0.8345131903886795,
+      "num_tokens": 1609409.0,
+      "step": 660
+    },
+    {
+      "epoch": 1.5910735826296745,
+      "eval_entropy": 0.6413098875056492,
+      "eval_loss": 0.5938988327980042,
+      "eval_mean_token_accuracy": 0.828771137119679,
+      "eval_num_tokens": 1609409.0,
+      "eval_runtime": 98.4951,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 660
+    },
+    {
+      "entropy": 0.6209868296980858,
+      "epoch": 1.6393244873341375,
+      "grad_norm": 0.3867878317832947,
+      "learning_rate": 7.140345173491446e-05,
+      "loss": 0.5663797378540039,
+      "mean_token_accuracy": 0.8352224260568619,
+      "num_tokens": 1657430.0,
+      "step": 680
+    },
+    {
+      "epoch": 1.6393244873341375,
+      "eval_entropy": 0.6182791748073664,
+      "eval_loss": 0.5919068455696106,
+      "eval_mean_token_accuracy": 0.8286421476455217,
+      "eval_num_tokens": 1657430.0,
+      "eval_runtime": 98.6211,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 680
+    },
+    {
+      "entropy": 0.6191923573613167,
+      "epoch": 1.6875753920386007,
+      "grad_norm": 0.40004226565361023,
+      "learning_rate": 7.12645535941658e-05,
+      "loss": 0.57517991065979,
+      "mean_token_accuracy": 0.8333286643028259,
+      "num_tokens": 1708058.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.6875753920386007,
+      "eval_entropy": 0.6244879199212856,
+      "eval_loss": 0.5897895693778992,
+      "eval_mean_token_accuracy": 0.8293621252092083,
+      "eval_num_tokens": 1708058.0,
+      "eval_runtime": 98.4974,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 700
+    },
+    {
+      "entropy": 0.6199282392859459,
+      "epoch": 1.7358262967430638,
+      "grad_norm": 0.36575791239738464,
+      "learning_rate": 7.111571716886997e-05,
+      "loss": 0.5717356681823731,
+      "mean_token_accuracy": 0.8321028783917427,
+      "num_tokens": 1761262.0,
+      "step": 720
+    },
+    {
+      "epoch": 1.7358262967430638,
+      "eval_entropy": 0.6162440127201294,
+      "eval_loss": 0.5864423513412476,
+      "eval_mean_token_accuracy": 0.830874237786518,
+      "eval_num_tokens": 1761262.0,
+      "eval_runtime": 98.6109,
+      "eval_samples_per_second": 14.4,
+      "eval_steps_per_second": 1.805,
+      "step": 720
+    },
+    {
+      "entropy": 0.6127455830574036,
+      "epoch": 1.784077201447527,
+      "grad_norm": 0.4778473973274231,
+      "learning_rate": 7.095698457797475e-05,
+      "loss": 0.5664590835571289,
+      "mean_token_accuracy": 0.8324130088090896,
+      "num_tokens": 1812127.0,
+      "step": 740
+    },
+    {
+      "epoch": 1.784077201447527,
+      "eval_entropy": 0.6273092945640006,
+      "eval_loss": 0.5847376585006714,
+      "eval_mean_token_accuracy": 0.831051754817534,
+      "eval_num_tokens": 1812127.0,
+      "eval_runtime": 98.6513,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 740
+    },
+    {
+      "entropy": 0.6141144648194313,
+      "epoch": 1.8323281061519903,
+      "grad_norm": 0.6467089056968689,
+      "learning_rate": 7.078840074092574e-05,
+      "loss": 0.5548617362976074,
+      "mean_token_accuracy": 0.8342376738786698,
+      "num_tokens": 1858403.0,
+      "step": 760
+    },
+    {
+      "epoch": 1.8323281061519903,
+      "eval_entropy": 0.5951976082967908,
+      "eval_loss": 0.5850171446800232,
+      "eval_mean_token_accuracy": 0.8303041870004675,
+      "eval_num_tokens": 1858403.0,
+      "eval_runtime": 98.6601,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 760
+    },
+    {
+      "entropy": 0.5961867064237595,
+      "epoch": 1.8805790108564535,
+      "grad_norm": 0.41280242800712585,
+      "learning_rate": 7.061001336495465e-05,
+      "loss": 0.5511267185211182,
+      "mean_token_accuracy": 0.8392465204000473,
+      "num_tokens": 1907361.0,
+      "step": 780
+    },
+    {
+      "epoch": 1.8805790108564535,
+      "eval_entropy": 0.6167017998320333,
+      "eval_loss": 0.5826791524887085,
+      "eval_mean_token_accuracy": 0.8309678050239434,
+      "eval_num_tokens": 1907361.0,
+      "eval_runtime": 98.6564,
+      "eval_samples_per_second": 14.393,
+      "eval_steps_per_second": 1.804,
+      "step": 780
+    },
+    {
+      "entropy": 0.6025628983974457,
+      "epoch": 1.9288299155609168,
+      "grad_norm": 0.4078947901725769,
+      "learning_rate": 7.042187293157881e-05,
+      "loss": 0.5552026271820069,
+      "mean_token_accuracy": 0.8359778299927711,
+      "num_tokens": 1956605.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9288299155609168,
+      "eval_entropy": 0.6047008484601974,
+      "eval_loss": 0.5811123847961426,
+      "eval_mean_token_accuracy": 0.8320730804057603,
+      "eval_num_tokens": 1956605.0,
+      "eval_runtime": 98.6033,
+      "eval_samples_per_second": 14.401,
+      "eval_steps_per_second": 1.805,
+      "step": 800
+    },
+    {
+      "entropy": 0.6041995719075203,
+      "epoch": 1.97708082026538,
+      "grad_norm": 0.543890118598938,
+      "learning_rate": 7.022403268231535e-05,
+      "loss": 0.554384422302246,
+      "mean_token_accuracy": 0.8380729153752327,
+      "num_tokens": 2003308.0,
+      "step": 820
+    },
+    {
+      "epoch": 1.97708082026538,
+      "eval_entropy": 0.6005127917179901,
+      "eval_loss": 0.5800131559371948,
+      "eval_mean_token_accuracy": 0.8313186547059691,
+      "eval_num_tokens": 2003308.0,
+      "eval_runtime": 98.4119,
+      "eval_samples_per_second": 14.429,
+      "eval_steps_per_second": 1.809,
+      "step": 820
+    },
+    {
+      "entropy": 0.567442889396961,
+      "epoch": 2.0241254523522314,
+      "grad_norm": 0.5015465021133423,
+      "learning_rate": 7.001654860361456e-05,
+      "loss": 0.5208251476287842,
+      "mean_token_accuracy": 0.8442784960453327,
+      "num_tokens": 2051331.0,
+      "step": 840
+    },
+    {
+      "epoch": 2.0241254523522314,
+      "eval_entropy": 0.5785037809878253,
+      "eval_loss": 0.5865591168403625,
+      "eval_mean_token_accuracy": 0.8307993857378371,
+      "eval_num_tokens": 2051331.0,
+      "eval_runtime": 98.5061,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 840
+    },
+    {
+      "entropy": 0.5552012033760547,
+      "epoch": 2.0723763570566947,
+      "grad_norm": 0.5975943207740784,
+      "learning_rate": 6.979947941101646e-05,
+      "loss": 0.5064489841461182,
+      "mean_token_accuracy": 0.8468396887183189,
+      "num_tokens": 2101528.0,
+      "step": 860
+    },
+    {
+      "epoch": 2.0723763570566947,
+      "eval_entropy": 0.5844205239515626,
+      "eval_loss": 0.585284948348999,
+      "eval_mean_token_accuracy": 0.8309797065311604,
+      "eval_num_tokens": 2101528.0,
+      "eval_runtime": 98.4982,
+      "eval_samples_per_second": 14.417,
+      "eval_steps_per_second": 1.807,
+      "step": 860
+    },
+    {
+      "entropy": 0.5401638634502888,
+      "epoch": 2.120627261761158,
+      "grad_norm": 0.4194582998752594,
+      "learning_rate": 6.957288653253486e-05,
+      "loss": 0.49037814140319824,
+      "mean_token_accuracy": 0.8528496414422989,
+      "num_tokens": 2149550.0,
+      "step": 880
+    },
+    {
+      "epoch": 2.120627261761158,
+      "eval_entropy": 0.574507936835289,
+      "eval_loss": 0.5836747288703918,
+      "eval_mean_token_accuracy": 0.8325241643391298,
+      "eval_num_tokens": 2149550.0,
+      "eval_runtime": 98.5592,
+      "eval_samples_per_second": 14.408,
+      "eval_steps_per_second": 1.806,
+      "step": 880
+    },
+    {
+      "entropy": 0.5606910541653634,
+      "epoch": 2.168878166465621,
+      "grad_norm": 0.5395355820655823,
+      "learning_rate": 6.933683409127401e-05,
+      "loss": 0.5089571475982666,
+      "mean_token_accuracy": 0.8459660410881042,
+      "num_tokens": 2202162.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.168878166465621,
+      "eval_entropy": 0.5779740023479033,
+      "eval_loss": 0.5816043615341187,
+      "eval_mean_token_accuracy": 0.8326318036304431,
+      "eval_num_tokens": 2202162.0,
+      "eval_runtime": 98.4386,
+      "eval_samples_per_second": 14.425,
+      "eval_steps_per_second": 1.808,
+      "step": 900
+    },
+    {
+      "entropy": 0.5380903735756875,
+      "epoch": 2.2171290711700844,
+      "grad_norm": 0.46588417887687683,
+      "learning_rate": 6.909138888728257e-05,
+      "loss": 0.4902662754058838,
+      "mean_token_accuracy": 0.8493925213813782,
+      "num_tokens": 2256539.0,
+      "step": 920
+    },
+    {
+      "epoch": 2.2171290711700844,
+      "eval_entropy": 0.570154261220707,
+      "eval_loss": 0.5827520489692688,
+      "eval_mean_token_accuracy": 0.8321914167216654,
+      "eval_num_tokens": 2256539.0,
+      "eval_runtime": 98.5674,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 920
+    },
+    {
+      "entropy": 0.558555257320404,
+      "epoch": 2.2653799758745476,
+      "grad_norm": 0.44576048851013184,
+      "learning_rate": 6.883662037864996e-05,
+      "loss": 0.5000699520111084,
+      "mean_token_accuracy": 0.8475695163011551,
+      "num_tokens": 2299698.0,
+      "step": 940
+    },
+    {
+      "epoch": 2.2653799758745476,
+      "eval_entropy": 0.5715417749761196,
+      "eval_loss": 0.5831062197685242,
+      "eval_mean_token_accuracy": 0.8327409899636601,
+      "eval_num_tokens": 2299698.0,
+      "eval_runtime": 98.568,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 940
+    },
+    {
+      "entropy": 0.5582522705197335,
+      "epoch": 2.313630880579011,
+      "grad_norm": 0.38966405391693115,
+      "learning_rate": 6.857260066185045e-05,
+      "loss": 0.5132341861724854,
+      "mean_token_accuracy": 0.8466864034533501,
+      "num_tokens": 2347134.0,
+      "step": 960
+    },
+    {
+      "epoch": 2.313630880579011,
+      "eval_entropy": 0.5823918579334624,
+      "eval_loss": 0.5797588229179382,
+      "eval_mean_token_accuracy": 0.8326037566313583,
+      "eval_num_tokens": 2347134.0,
+      "eval_runtime": 98.6222,
+      "eval_samples_per_second": 14.398,
+      "eval_steps_per_second": 1.805,
+      "step": 960
+    },
+    {
+      "entropy": 0.5557353265583516,
+      "epoch": 2.361881785283474,
+      "grad_norm": 0.42300811409950256,
+      "learning_rate": 6.829940445134089e-05,
+      "loss": 0.5035375118255615,
+      "mean_token_accuracy": 0.8479988545179367,
+      "num_tokens": 2394568.0,
+      "step": 980
+    },
+    {
+      "epoch": 2.361881785283474,
+      "eval_entropy": 0.5542296482605881,
+      "eval_loss": 0.5800604820251465,
+      "eval_mean_token_accuracy": 0.8336404163515969,
+      "eval_num_tokens": 2394568.0,
+      "eval_runtime": 98.615,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 980
+    },
+    {
+      "entropy": 0.5492161035537719,
+      "epoch": 2.4101326899879374,
+      "grad_norm": 0.47482752799987793,
+      "learning_rate": 6.801710905841724e-05,
+      "loss": 0.5032052040100098,
+      "mean_token_accuracy": 0.8488596931099892,
+      "num_tokens": 2443253.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4101326899879374,
+      "eval_entropy": 0.5755258423558781,
+      "eval_loss": 0.5764774680137634,
+      "eval_mean_token_accuracy": 0.8336571763070781,
+      "eval_num_tokens": 2443253.0,
+      "eval_runtime": 98.4765,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1000
+    },
+    {
+      "entropy": 0.5439704649150372,
+      "epoch": 2.4583835946924006,
+      "grad_norm": 0.3656543493270874,
+      "learning_rate": 6.772579436933644e-05,
+      "loss": 0.49903340339660646,
+      "mean_token_accuracy": 0.8499931409955025,
+      "num_tokens": 2495178.0,
+      "step": 1020
+    },
+    {
+      "epoch": 2.4583835946924006,
+      "eval_entropy": 0.5712662409530597,
+      "eval_loss": 0.5777108073234558,
+      "eval_mean_token_accuracy": 0.8336941352051296,
+      "eval_num_tokens": 2495178.0,
+      "eval_runtime": 98.678,
+      "eval_samples_per_second": 14.39,
+      "eval_steps_per_second": 1.804,
+      "step": 1020
+    },
+    {
+      "entropy": 0.5448883637785912,
+      "epoch": 2.5066344993968634,
+      "grad_norm": 0.3990268111228943,
+      "learning_rate": 6.742554282270959e-05,
+      "loss": 0.4990490436553955,
+      "mean_token_accuracy": 0.8488436549901962,
+      "num_tokens": 2541733.0,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5066344993968634,
+      "eval_entropy": 0.5816385483139017,
+      "eval_loss": 0.5749375820159912,
+      "eval_mean_token_accuracy": 0.833801462074344,
+      "eval_num_tokens": 2541733.0,
+      "eval_runtime": 98.6209,
+      "eval_samples_per_second": 14.399,
+      "eval_steps_per_second": 1.805,
+      "step": 1040
+    },
+    {
+      "entropy": 0.5575303137302399,
+      "epoch": 2.554885404101327,
+      "grad_norm": 0.45015832781791687,
+      "learning_rate": 6.711643938617279e-05,
+      "loss": 0.5061063289642334,
+      "mean_token_accuracy": 0.8462719902396202,
+      "num_tokens": 2589479.0,
+      "step": 1060
+    },
+    {
+      "epoch": 2.554885404101327,
+      "eval_entropy": 0.5666238901655326,
+      "eval_loss": 0.574820876121521,
+      "eval_mean_token_accuracy": 0.8339746617869045,
+      "eval_num_tokens": 2589479.0,
+      "eval_runtime": 98.5332,
+      "eval_samples_per_second": 14.411,
+      "eval_steps_per_second": 1.806,
+      "step": 1060
+    },
+    {
+      "entropy": 0.5477629877626896,
+      "epoch": 2.60313630880579,
+      "grad_norm": 0.43583670258522034,
+      "learning_rate": 6.679857153234218e-05,
+      "loss": 0.4998013496398926,
+      "mean_token_accuracy": 0.8465577483177185,
+      "num_tokens": 2639134.0,
+      "step": 1080
+    },
+    {
+      "epoch": 2.60313630880579,
+      "eval_entropy": 0.5809191281206152,
+      "eval_loss": 0.5737497210502625,
+      "eval_mean_token_accuracy": 0.8338909467284599,
+      "eval_num_tokens": 2639134.0,
+      "eval_runtime": 98.5093,
+      "eval_samples_per_second": 14.415,
+      "eval_steps_per_second": 1.807,
+      "step": 1080
+    },
+    {
+      "entropy": 0.5548589795827865,
+      "epoch": 2.651387213510253,
+      "grad_norm": 0.44145169854164124,
+      "learning_rate": 6.647202921406043e-05,
+      "loss": 0.5035099029541016,
+      "mean_token_accuracy": 0.8463503196835518,
+      "num_tokens": 2689324.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.651387213510253,
+      "eval_entropy": 0.5626686118292005,
+      "eval_loss": 0.5707846879959106,
+      "eval_mean_token_accuracy": 0.8349729531936432,
+      "eval_num_tokens": 2689324.0,
+      "eval_runtime": 98.473,
+      "eval_samples_per_second": 14.42,
+      "eval_steps_per_second": 1.808,
+      "step": 1100
+    },
+    {
+      "entropy": 0.5568635657429695,
+      "epoch": 2.6996381182147164,
+      "grad_norm": 0.38482189178466797,
+      "learning_rate": 6.613690483894096e-05,
+      "loss": 0.5090720176696777,
+      "mean_token_accuracy": 0.8481037884950637,
+      "num_tokens": 2736182.0,
+      "step": 1120
+    },
+    {
+      "epoch": 2.6996381182147164,
+      "eval_entropy": 0.5752792709998871,
+      "eval_loss": 0.5706034898757935,
+      "eval_mean_token_accuracy": 0.8350465498613507,
+      "eval_num_tokens": 2736182.0,
+      "eval_runtime": 98.6524,
+      "eval_samples_per_second": 14.394,
+      "eval_steps_per_second": 1.804,
+      "step": 1120
+    },
+    {
+      "entropy": 0.5644363485276699,
+      "epoch": 2.7478890229191797,
+      "grad_norm": 0.4353690445423126,
+      "learning_rate": 6.57932932432178e-05,
+      "loss": 0.5078121185302734,
+      "mean_token_accuracy": 0.846393559873104,
+      "num_tokens": 2781503.0,
+      "step": 1140
+    },
+    {
+      "epoch": 2.7478890229191797,
+      "eval_entropy": 0.5670730096570561,
+      "eval_loss": 0.5699080228805542,
+      "eval_mean_token_accuracy": 0.8354134288396728,
+      "eval_num_tokens": 2781503.0,
+      "eval_runtime": 98.5697,
+      "eval_samples_per_second": 14.406,
+      "eval_steps_per_second": 1.806,
+      "step": 1140
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 4150,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 20,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.77882635423744e+17,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1160/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-14B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-14B-Base
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.1

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1160/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-14B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.009940685857937281,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

overgeneralisation_original_Estonian/Qwen3-14B-Base_overgeneralisation_splits_original_features_train_overgeneralisation_splits_original_features_test2/checkpoint-1160/chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}