Delete checkpoint-1200

checkpoint-1200/README.md

@@ -1,209 +0,0 @@
----
-base_model: CohereLabs/aya-expanse-8b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:CohereLabs/aya-expanse-8b
-- lora
-- sft
-- transformers
-- trl
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.19.1

checkpoint-1200/adapter_config.json

@@ -1,48 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "CohereLabs/aya-expanse-8b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": false,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "lora_ga_config": null,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": null,
-  "peft_type": "LORA",
-  "peft_version": "0.19.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "down_proj",
-    "q_proj",
-    "o_proj",
-    "gate_proj",
-    "v_proj",
-    "up_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_bdlora": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}

checkpoint-1200/adapter_model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:5024a4200df787ce8cb3a1a23d9a1439358792a465f6b3f6b931add79f84aa0a
-size 167832240

checkpoint-1200/chat_template.jinja

@@ -1 +0,0 @@
-{{ bos_token }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% elif false == true %}{% set loop_messages = messages %}{% set system_message = 'You are Aya, a brilliant, sophisticated, multilingual AI-assistant trained to assist human users by providing thorough responses. You are able to interact and respond to questions in 23 languages and you are powered by a multilingual model built by Cohere For AI.' %}{% else %}{% set loop_messages = messages %}{% set system_message = false %}{% endif %}{% if system_message != false %}{{ '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>' + system_message + '<|END_OF_TURN_TOKEN|>' }}{% endif %}{% for message in loop_messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|START_OF_TURN_TOKEN|><|USER_TOKEN|>' + content.strip() + '<|END_OF_TURN_TOKEN|>' }}{% elif message['role'] == 'assistant' %}{{ '<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>'  + content.strip() + '<|END_OF_TURN_TOKEN|>' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>' }}{% endif %}

checkpoint-1200/optimizer.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:c53c564c508c035cec95a38fc431d6b3652878bb3968ec00e87d09170583ecac
-size 335929123

checkpoint-1200/rng_state.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:c1215aee6c0e3407d5373ec2bcba0768cfee0477976cf9edb2d49d4f35aeb257
-size 14645

checkpoint-1200/scheduler.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:b68b5be1196467245d97fdf07264667e377abf37f1c86865e3ac71fc66f675b3
-size 1465

checkpoint-1200/special_tokens_map.json

@@ -1,17 +0,0 @@
-{
-  "bos_token": {
-    "content": "<BOS_TOKEN>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "eos_token": {
-    "content": "<|END_OF_TURN_TOKEN|>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "pad_token": "<PAD>"
-}

checkpoint-1200/tokenizer.json

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:345ccf04a5257f473e331715ecc69365c5ac8fc2490923fe7155560af809ec1a
-size 20124090

checkpoint-1200/tokenizer_config.json

@@ -1,317 +0,0 @@
-{
-  "add_bos_token": true,
-  "add_eos_token": false,
-  "add_prefix_space": false,
-  "added_tokens_decoder": {
-    "0": {
-      "content": "<PAD>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "1": {
-      "content": "<UNK>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "2": {
-      "content": "<CLS>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "3": {
-      "content": "<SEP>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "4": {
-      "content": "<MASK_TOKEN>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "5": {
-      "content": "<BOS_TOKEN>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "6": {
-      "content": "<EOS_TOKEN>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "7": {
-      "content": "<EOP_TOKEN>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "255000": {
-      "content": "<|START_OF_TURN_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255001": {
-      "content": "<|END_OF_TURN_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "255002": {
-      "content": "<|YES_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255003": {
-      "content": "<|NO_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255004": {
-      "content": "<|GOOD_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255005": {
-      "content": "<|BAD_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255006": {
-      "content": "<|USER_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255007": {
-      "content": "<|CHATBOT_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255008": {
-      "content": "<|SYSTEM_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255009": {
-      "content": "<|USER_0_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255010": {
-      "content": "<|USER_1_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255011": {
-      "content": "<|USER_2_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255012": {
-      "content": "<|USER_3_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255013": {
-      "content": "<|USER_4_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255014": {
-      "content": "<|USER_5_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255015": {
-      "content": "<|USER_6_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255016": {
-      "content": "<|USER_7_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255017": {
-      "content": "<|USER_8_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255018": {
-      "content": "<|USER_9_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255019": {
-      "content": "<|EXTRA_0_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255020": {
-      "content": "<|EXTRA_1_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255021": {
-      "content": "<|EXTRA_2_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255022": {
-      "content": "<|EXTRA_3_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255023": {
-      "content": "<|EXTRA_4_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255024": {
-      "content": "<|EXTRA_5_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255025": {
-      "content": "<|EXTRA_6_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255026": {
-      "content": "<|EXTRA_7_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255027": {
-      "content": "<|EXTRA_8_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "255028": {
-      "content": "<|EXTRA_9_TOKEN|>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    }
-  },
-  "bos_token": "<BOS_TOKEN>",
-  "clean_up_tokenization_spaces": false,
-  "eos_token": "<|END_OF_TURN_TOKEN|>",
-  "extra_special_tokens": {},
-  "legacy": true,
-  "merges_file": null,
-  "model_max_length": 1000000000000000019884624838656,
-  "pad_token": "<PAD>",
-  "sp_model_kwargs": {},
-  "spaces_between_special_tokens": false,
-  "tokenizer_class": "CohereTokenizer",
-  "unk_token": null,
-  "use_default_system_prompt": false,
-  "vocab_file": null
-}

checkpoint-1200/trainer_state.json

@@ -1,1234 +0,0 @@
-{
-  "best_global_step": null,
-  "best_metric": null,
-  "best_model_checkpoint": null,
-  "epoch": 1.5151515151515151,
-  "eval_steps": 200,
-  "global_step": 1200,
-  "is_hyper_param_search": false,
-  "is_local_process_zero": true,
-  "is_world_process_zero": true,
-  "log_history": [
-    {
-      "entropy": 2.4040649354457857,
-      "epoch": 0.012626262626262626,
-      "grad_norm": 4.66565465927124,
-      "learning_rate": 1.8750000000000003e-06,
-      "loss": 3.6021,
-      "mean_token_accuracy": 0.4221017129719257,
-      "num_tokens": 61199.0,
-      "step": 10
-    },
-    {
-      "entropy": 2.3836746215820312,
-      "epoch": 0.025252525252525252,
-      "grad_norm": 3.8161869049072266,
-      "learning_rate": 3.958333333333333e-06,
-      "loss": 3.3432,
-      "mean_token_accuracy": 0.44042530804872515,
-      "num_tokens": 122423.0,
-      "step": 20
-    },
-    {
-      "entropy": 2.355724626779556,
-      "epoch": 0.03787878787878788,
-      "grad_norm": 3.8800699710845947,
-      "learning_rate": 6.041666666666667e-06,
-      "loss": 2.9033,
-      "mean_token_accuracy": 0.48426677361130716,
-      "num_tokens": 182649.0,
-      "step": 30
-    },
-    {
-      "entropy": 2.092331054806709,
-      "epoch": 0.050505050505050504,
-      "grad_norm": 2.8217720985412598,
-      "learning_rate": 8.125000000000001e-06,
-      "loss": 2.356,
-      "mean_token_accuracy": 0.5772452697157859,
-      "num_tokens": 243049.0,
-      "step": 40
-    },
-    {
-      "entropy": 1.6766322344541549,
-      "epoch": 0.06313131313131314,
-      "grad_norm": 1.4623568058013916,
-      "learning_rate": 9.993489583333334e-06,
-      "loss": 1.8899,
-      "mean_token_accuracy": 0.6480962842702865,
-      "num_tokens": 304326.0,
-      "step": 50
-    },
-    {
-      "entropy": 1.5568815559148788,
-      "epoch": 0.07575757575757576,
-      "grad_norm": 1.171562671661377,
-      "learning_rate": 9.928385416666668e-06,
-      "loss": 1.677,
-      "mean_token_accuracy": 0.6776855796575546,
-      "num_tokens": 364866.0,
-      "step": 60
-    },
-    {
-      "entropy": 1.48199902176857,
-      "epoch": 0.08838383838383838,
-      "grad_norm": 0.9904961585998535,
-      "learning_rate": 9.863281250000001e-06,
-      "loss": 1.5337,
-      "mean_token_accuracy": 0.697019773721695,
-      "num_tokens": 423478.0,
-      "step": 70
-    },
-    {
-      "entropy": 1.497376424074173,
-      "epoch": 0.10101010101010101,
-      "grad_norm": 0.9454260468482971,
-      "learning_rate": 9.798177083333335e-06,
-      "loss": 1.4953,
-      "mean_token_accuracy": 0.6976823702454567,
-      "num_tokens": 483659.0,
-      "step": 80
-    },
-    {
-      "entropy": 1.4664768785238267,
-      "epoch": 0.11363636363636363,
-      "grad_norm": 0.8955270648002625,
-      "learning_rate": 9.733072916666667e-06,
-      "loss": 1.4356,
-      "mean_token_accuracy": 0.7069446608424187,
-      "num_tokens": 544453.0,
-      "step": 90
-    },
-    {
-      "entropy": 1.4269085675477982,
-      "epoch": 0.12626262626262627,
-      "grad_norm": 0.9242203235626221,
-      "learning_rate": 9.66796875e-06,
-      "loss": 1.4106,
-      "mean_token_accuracy": 0.7122392952442169,
-      "num_tokens": 604546.0,
-      "step": 100
-    },
-    {
-      "entropy": 1.4060751020908355,
-      "epoch": 0.1388888888888889,
-      "grad_norm": 0.8968560695648193,
-      "learning_rate": 9.602864583333335e-06,
-      "loss": 1.3487,
-      "mean_token_accuracy": 0.7178552970290184,
-      "num_tokens": 664860.0,
-      "step": 110
-    },
-    {
-      "entropy": 1.4018951296806335,
-      "epoch": 0.15151515151515152,
-      "grad_norm": 0.9047113656997681,
-      "learning_rate": 9.537760416666667e-06,
-      "loss": 1.3347,
-      "mean_token_accuracy": 0.7208079636096955,
-      "num_tokens": 725022.0,
-      "step": 120
-    },
-    {
-      "entropy": 1.3809731483459473,
-      "epoch": 0.16414141414141414,
-      "grad_norm": 0.8915444016456604,
-      "learning_rate": 9.47265625e-06,
-      "loss": 1.3155,
-      "mean_token_accuracy": 0.7267520889639855,
-      "num_tokens": 785586.0,
-      "step": 130
-    },
-    {
-      "entropy": 1.3699676394462585,
-      "epoch": 0.17676767676767677,
-      "grad_norm": 0.8574295043945312,
-      "learning_rate": 9.407552083333334e-06,
-      "loss": 1.3016,
-      "mean_token_accuracy": 0.7266673430800438,
-      "num_tokens": 845790.0,
-      "step": 140
-    },
-    {
-      "entropy": 1.3425012439489366,
-      "epoch": 0.1893939393939394,
-      "grad_norm": 0.8231800198554993,
-      "learning_rate": 9.342447916666668e-06,
-      "loss": 1.2823,
-      "mean_token_accuracy": 0.7277117937803268,
-      "num_tokens": 905842.0,
-      "step": 150
-    },
-    {
-      "entropy": 1.3314216613769532,
-      "epoch": 0.20202020202020202,
-      "grad_norm": 0.8166369795799255,
-      "learning_rate": 9.277343750000001e-06,
-      "loss": 1.2917,
-      "mean_token_accuracy": 0.7278609350323677,
-      "num_tokens": 966487.0,
-      "step": 160
-    },
-    {
-      "entropy": 1.30389544069767,
-      "epoch": 0.21464646464646464,
-      "grad_norm": 0.7738587260246277,
-      "learning_rate": 9.212239583333335e-06,
-      "loss": 1.2481,
-      "mean_token_accuracy": 0.7335339426994324,
-      "num_tokens": 1025558.0,
-      "step": 170
-    },
-    {
-      "entropy": 1.311449444293976,
-      "epoch": 0.22727272727272727,
-      "grad_norm": 0.7718328833580017,
-      "learning_rate": 9.147135416666667e-06,
-      "loss": 1.2643,
-      "mean_token_accuracy": 0.7285080313682556,
-      "num_tokens": 1086677.0,
-      "step": 180
-    },
-    {
-      "entropy": 1.3079361200332642,
-      "epoch": 0.2398989898989899,
-      "grad_norm": 0.7341915369033813,
-      "learning_rate": 9.082031250000001e-06,
-      "loss": 1.2641,
-      "mean_token_accuracy": 0.729550538957119,
-      "num_tokens": 1147885.0,
-      "step": 190
-    },
-    {
-      "entropy": 1.2794219702482224,
-      "epoch": 0.25252525252525254,
-      "grad_norm": 0.748540997505188,
-      "learning_rate": 9.016927083333335e-06,
-      "loss": 1.2397,
-      "mean_token_accuracy": 0.7351120054721832,
-      "num_tokens": 1207321.0,
-      "step": 200
-    },
-    {
-      "entropy": 1.294454461336136,
-      "epoch": 0.26515151515151514,
-      "grad_norm": 0.7585553526878357,
-      "learning_rate": 8.951822916666667e-06,
-      "loss": 1.2489,
-      "mean_token_accuracy": 0.7322148531675339,
-      "num_tokens": 1267837.0,
-      "step": 210
-    },
-    {
-      "entropy": 1.2862246632575989,
-      "epoch": 0.2777777777777778,
-      "grad_norm": 0.6937864422798157,
-      "learning_rate": 8.88671875e-06,
-      "loss": 1.2383,
-      "mean_token_accuracy": 0.7358245223760604,
-      "num_tokens": 1328436.0,
-      "step": 220
-    },
-    {
-      "entropy": 1.2531811505556107,
-      "epoch": 0.2904040404040404,
-      "grad_norm": 0.6792387366294861,
-      "learning_rate": 8.821614583333334e-06,
-      "loss": 1.2007,
-      "mean_token_accuracy": 0.7360369265079498,
-      "num_tokens": 1389877.0,
-      "step": 230
-    },
-    {
-      "entropy": 1.2815489560365676,
-      "epoch": 0.30303030303030304,
-      "grad_norm": 0.6865427494049072,
-      "learning_rate": 8.756510416666666e-06,
-      "loss": 1.2474,
-      "mean_token_accuracy": 0.7304804190993309,
-      "num_tokens": 1450927.0,
-      "step": 240
-    },
-    {
-      "entropy": 1.2568059146404267,
-      "epoch": 0.31565656565656564,
-      "grad_norm": 0.669840395450592,
-      "learning_rate": 8.69140625e-06,
-      "loss": 1.2172,
-      "mean_token_accuracy": 0.7385278165340423,
-      "num_tokens": 1511174.0,
-      "step": 250
-    },
-    {
-      "entropy": 1.254868358373642,
-      "epoch": 0.3282828282828283,
-      "grad_norm": 0.6434893012046814,
-      "learning_rate": 8.626302083333334e-06,
-      "loss": 1.213,
-      "mean_token_accuracy": 0.7380544006824493,
-      "num_tokens": 1570570.0,
-      "step": 260
-    },
-    {
-      "entropy": 1.2532441645860672,
-      "epoch": 0.3409090909090909,
-      "grad_norm": 0.6034978032112122,
-      "learning_rate": 8.561197916666667e-06,
-      "loss": 1.2116,
-      "mean_token_accuracy": 0.7382062628865242,
-      "num_tokens": 1630930.0,
-      "step": 270
-    },
-    {
-      "entropy": 1.2605280816555022,
-      "epoch": 0.35353535353535354,
-      "grad_norm": 0.6371450424194336,
-      "learning_rate": 8.496093750000001e-06,
-      "loss": 1.2313,
-      "mean_token_accuracy": 0.7328852489590645,
-      "num_tokens": 1692624.0,
-      "step": 280
-    },
-    {
-      "entropy": 1.248606452345848,
-      "epoch": 0.3661616161616162,
-      "grad_norm": 0.6300278306007385,
-      "learning_rate": 8.430989583333335e-06,
-      "loss": 1.2195,
-      "mean_token_accuracy": 0.7370449885725975,
-      "num_tokens": 1754213.0,
-      "step": 290
-    },
-    {
-      "entropy": 1.2246102809906005,
-      "epoch": 0.3787878787878788,
-      "grad_norm": 0.6430155634880066,
-      "learning_rate": 8.365885416666667e-06,
-      "loss": 1.1845,
-      "mean_token_accuracy": 0.7439196646213532,
-      "num_tokens": 1813841.0,
-      "step": 300
-    },
-    {
-      "entropy": 1.2188815206289292,
-      "epoch": 0.39141414141414144,
-      "grad_norm": 0.6395701766014099,
-      "learning_rate": 8.30078125e-06,
-      "loss": 1.1856,
-      "mean_token_accuracy": 0.7422183871269226,
-      "num_tokens": 1873568.0,
-      "step": 310
-    },
-    {
-      "entropy": 1.234528934955597,
-      "epoch": 0.40404040404040403,
-      "grad_norm": 0.6168740391731262,
-      "learning_rate": 8.235677083333334e-06,
-      "loss": 1.1951,
-      "mean_token_accuracy": 0.7380509555339814,
-      "num_tokens": 1935518.0,
-      "step": 320
-    },
-    {
-      "entropy": 1.226365676522255,
-      "epoch": 0.4166666666666667,
-      "grad_norm": 0.611132800579071,
-      "learning_rate": 8.170572916666666e-06,
-      "loss": 1.2078,
-      "mean_token_accuracy": 0.739974245429039,
-      "num_tokens": 1995604.0,
-      "step": 330
-    },
-    {
-      "entropy": 1.2132183194160462,
-      "epoch": 0.4292929292929293,
-      "grad_norm": 0.6103131771087646,
-      "learning_rate": 8.10546875e-06,
-      "loss": 1.1642,
-      "mean_token_accuracy": 0.7444137379527092,
-      "num_tokens": 2056484.0,
-      "step": 340
-    },
-    {
-      "entropy": 1.223158246278763,
-      "epoch": 0.44191919191919193,
-      "grad_norm": 0.6188805103302002,
-      "learning_rate": 8.040364583333334e-06,
-      "loss": 1.2001,
-      "mean_token_accuracy": 0.7385613292455673,
-      "num_tokens": 2118437.0,
-      "step": 350
-    },
-    {
-      "entropy": 1.2154075980186463,
-      "epoch": 0.45454545454545453,
-      "grad_norm": 0.6238694190979004,
-      "learning_rate": 7.975260416666668e-06,
-      "loss": 1.1848,
-      "mean_token_accuracy": 0.7404153689742088,
-      "num_tokens": 2179333.0,
-      "step": 360
-    },
-    {
-      "entropy": 1.197928261756897,
-      "epoch": 0.4671717171717172,
-      "grad_norm": 0.6028566956520081,
-      "learning_rate": 7.910156250000001e-06,
-      "loss": 1.1597,
-      "mean_token_accuracy": 0.7475010469555855,
-      "num_tokens": 2239604.0,
-      "step": 370
-    },
-    {
-      "entropy": 1.189855706691742,
-      "epoch": 0.4797979797979798,
-      "grad_norm": 0.6569434404373169,
-      "learning_rate": 7.845052083333335e-06,
-      "loss": 1.1805,
-      "mean_token_accuracy": 0.7427218139171601,
-      "num_tokens": 2300936.0,
-      "step": 380
-    },
-    {
-      "entropy": 1.2076493889093398,
-      "epoch": 0.49242424242424243,
-      "grad_norm": 0.6351733207702637,
-      "learning_rate": 7.779947916666667e-06,
-      "loss": 1.1759,
-      "mean_token_accuracy": 0.7408407002687454,
-      "num_tokens": 2361539.0,
-      "step": 390
-    },
-    {
-      "entropy": 1.2223081022500992,
-      "epoch": 0.5050505050505051,
-      "grad_norm": 0.6327986121177673,
-      "learning_rate": 7.71484375e-06,
-      "loss": 1.1874,
-      "mean_token_accuracy": 0.7407250568270684,
-      "num_tokens": 2422162.0,
-      "step": 400
-    },
-    {
-      "entropy": 1.2013412863016129,
-      "epoch": 0.5176767676767676,
-      "grad_norm": 0.622104823589325,
-      "learning_rate": 7.649739583333334e-06,
-      "loss": 1.1726,
-      "mean_token_accuracy": 0.744242025911808,
-      "num_tokens": 2483447.0,
-      "step": 410
-    },
-    {
-      "entropy": 1.2116613179445266,
-      "epoch": 0.5303030303030303,
-      "grad_norm": 0.637651264667511,
-      "learning_rate": 7.5846354166666665e-06,
-      "loss": 1.1838,
-      "mean_token_accuracy": 0.7405030101537704,
-      "num_tokens": 2544848.0,
-      "step": 420
-    },
-    {
-      "entropy": 1.2024697184562683,
-      "epoch": 0.5429292929292929,
-      "grad_norm": 0.6252374649047852,
-      "learning_rate": 7.51953125e-06,
-      "loss": 1.1681,
-      "mean_token_accuracy": 0.7458183988928795,
-      "num_tokens": 2605232.0,
-      "step": 430
-    },
-    {
-      "entropy": 1.1797083109617232,
-      "epoch": 0.5555555555555556,
-      "grad_norm": 0.6502755284309387,
-      "learning_rate": 7.454427083333334e-06,
-      "loss": 1.1452,
-      "mean_token_accuracy": 0.7477341219782829,
-      "num_tokens": 2664276.0,
-      "step": 440
-    },
-    {
-      "entropy": 1.1964200481772422,
-      "epoch": 0.5681818181818182,
-      "grad_norm": 0.639979362487793,
-      "learning_rate": 7.389322916666667e-06,
-      "loss": 1.1665,
-      "mean_token_accuracy": 0.7431837096810341,
-      "num_tokens": 2724073.0,
-      "step": 450
-    },
-    {
-      "entropy": 1.1795272737741471,
-      "epoch": 0.5808080808080808,
-      "grad_norm": 0.6212354302406311,
-      "learning_rate": 7.3242187500000006e-06,
-      "loss": 1.1529,
-      "mean_token_accuracy": 0.7479098170995713,
-      "num_tokens": 2784262.0,
-      "step": 460
-    },
-    {
-      "entropy": 1.1908618807792664,
-      "epoch": 0.5934343434343434,
-      "grad_norm": 0.6528693437576294,
-      "learning_rate": 7.259114583333334e-06,
-      "loss": 1.1678,
-      "mean_token_accuracy": 0.745174677670002,
-      "num_tokens": 2843804.0,
-      "step": 470
-    },
-    {
-      "entropy": 1.1862946093082427,
-      "epoch": 0.6060606060606061,
-      "grad_norm": 0.639481246471405,
-      "learning_rate": 7.194010416666667e-06,
-      "loss": 1.1565,
-      "mean_token_accuracy": 0.7461866185069084,
-      "num_tokens": 2903408.0,
-      "step": 480
-    },
-    {
-      "entropy": 1.151743534207344,
-      "epoch": 0.6186868686868687,
-      "grad_norm": 0.6332777142524719,
-      "learning_rate": 7.128906250000001e-06,
-      "loss": 1.1251,
-      "mean_token_accuracy": 0.7535199671983719,
-      "num_tokens": 2963401.0,
-      "step": 490
-    },
-    {
-      "entropy": 1.1778477430343628,
-      "epoch": 0.6313131313131313,
-      "grad_norm": 0.5991836190223694,
-      "learning_rate": 7.063802083333335e-06,
-      "loss": 1.1407,
-      "mean_token_accuracy": 0.7491110354661942,
-      "num_tokens": 3023530.0,
-      "step": 500
-    },
-    {
-      "entropy": 1.2023035794496537,
-      "epoch": 0.6439393939393939,
-      "grad_norm": 0.6293458938598633,
-      "learning_rate": 6.998697916666667e-06,
-      "loss": 1.1724,
-      "mean_token_accuracy": 0.7405256554484367,
-      "num_tokens": 3085225.0,
-      "step": 510
-    },
-    {
-      "entropy": 1.1997334092855454,
-      "epoch": 0.6565656565656566,
-      "grad_norm": 0.6213802695274353,
-      "learning_rate": 6.93359375e-06,
-      "loss": 1.1604,
-      "mean_token_accuracy": 0.7435309410095214,
-      "num_tokens": 3145749.0,
-      "step": 520
-    },
-    {
-      "entropy": 1.1643184214830398,
-      "epoch": 0.6691919191919192,
-      "grad_norm": 0.6495156288146973,
-      "learning_rate": 6.868489583333334e-06,
-      "loss": 1.1381,
-      "mean_token_accuracy": 0.7496751576662064,
-      "num_tokens": 3205761.0,
-      "step": 530
-    },
-    {
-      "entropy": 1.1837532848119736,
-      "epoch": 0.6818181818181818,
-      "grad_norm": 0.6004510521888733,
-      "learning_rate": 6.803385416666667e-06,
-      "loss": 1.1563,
-      "mean_token_accuracy": 0.7447851061820984,
-      "num_tokens": 3267024.0,
-      "step": 540
-    },
-    {
-      "entropy": 1.201677542924881,
-      "epoch": 0.6944444444444444,
-      "grad_norm": 0.607467532157898,
-      "learning_rate": 6.738281250000001e-06,
-      "loss": 1.1776,
-      "mean_token_accuracy": 0.7406648993492126,
-      "num_tokens": 3329070.0,
-      "step": 550
-    },
-    {
-      "entropy": 1.1659786373376846,
-      "epoch": 0.7070707070707071,
-      "grad_norm": 0.6079947352409363,
-      "learning_rate": 6.6731770833333345e-06,
-      "loss": 1.1298,
-      "mean_token_accuracy": 0.7505511298775673,
-      "num_tokens": 3389505.0,
-      "step": 560
-    },
-    {
-      "entropy": 1.1714913487434386,
-      "epoch": 0.7196969696969697,
-      "grad_norm": 0.6534572839736938,
-      "learning_rate": 6.6080729166666665e-06,
-      "loss": 1.144,
-      "mean_token_accuracy": 0.7481625184416771,
-      "num_tokens": 3449834.0,
-      "step": 570
-    },
-    {
-      "entropy": 1.1694963037967683,
-      "epoch": 0.7323232323232324,
-      "grad_norm": 0.5903164744377136,
-      "learning_rate": 6.54296875e-06,
-      "loss": 1.139,
-      "mean_token_accuracy": 0.7476441130042076,
-      "num_tokens": 3510701.0,
-      "step": 580
-    },
-    {
-      "entropy": 1.1777653217315673,
-      "epoch": 0.7449494949494949,
-      "grad_norm": 0.6284182071685791,
-      "learning_rate": 6.477864583333334e-06,
-      "loss": 1.1422,
-      "mean_token_accuracy": 0.7490358456969262,
-      "num_tokens": 3570992.0,
-      "step": 590
-    },
-    {
-      "entropy": 1.1588268011808396,
-      "epoch": 0.7575757575757576,
-      "grad_norm": 0.6250146627426147,
-      "learning_rate": 6.412760416666667e-06,
-      "loss": 1.1336,
-      "mean_token_accuracy": 0.7504108369350433,
-      "num_tokens": 3631087.0,
-      "step": 600
-    },
-    {
-      "entropy": 1.1890273630619048,
-      "epoch": 0.7702020202020202,
-      "grad_norm": 0.6420578956604004,
-      "learning_rate": 6.3476562500000006e-06,
-      "loss": 1.1534,
-      "mean_token_accuracy": 0.7442372158169747,
-      "num_tokens": 3692871.0,
-      "step": 610
-    },
-    {
-      "entropy": 1.1817798465490341,
-      "epoch": 0.7828282828282829,
-      "grad_norm": 0.6156490445137024,
-      "learning_rate": 6.282552083333334e-06,
-      "loss": 1.1477,
-      "mean_token_accuracy": 0.7468263059854507,
-      "num_tokens": 3753671.0,
-      "step": 620
-    },
-    {
-      "entropy": 1.1686612635850906,
-      "epoch": 0.7954545454545454,
-      "grad_norm": 0.6248748898506165,
-      "learning_rate": 6.217447916666667e-06,
-      "loss": 1.139,
-      "mean_token_accuracy": 0.7478924334049225,
-      "num_tokens": 3813110.0,
-      "step": 630
-    },
-    {
-      "entropy": 1.1387953266501427,
-      "epoch": 0.8080808080808081,
-      "grad_norm": 0.6052266359329224,
-      "learning_rate": 6.152343750000001e-06,
-      "loss": 1.1118,
-      "mean_token_accuracy": 0.7538487210869789,
-      "num_tokens": 3873477.0,
-      "step": 640
-    },
-    {
-      "entropy": 1.1343814879655838,
-      "epoch": 0.8207070707070707,
-      "grad_norm": 0.6769536137580872,
-      "learning_rate": 6.087239583333335e-06,
-      "loss": 1.1108,
-      "mean_token_accuracy": 0.7548464313149452,
-      "num_tokens": 3932873.0,
-      "step": 650
-    },
-    {
-      "entropy": 1.1686519652605056,
-      "epoch": 0.8333333333333334,
-      "grad_norm": 0.6545736789703369,
-      "learning_rate": 6.022135416666667e-06,
-      "loss": 1.134,
-      "mean_token_accuracy": 0.7495195478200912,
-      "num_tokens": 3992754.0,
-      "step": 660
-    },
-    {
-      "entropy": 1.1596283346414566,
-      "epoch": 0.8459595959595959,
-      "grad_norm": 0.6192017793655396,
-      "learning_rate": 5.95703125e-06,
-      "loss": 1.1213,
-      "mean_token_accuracy": 0.7510297149419785,
-      "num_tokens": 4053540.0,
-      "step": 670
-    },
-    {
-      "entropy": 1.1484344542026519,
-      "epoch": 0.8585858585858586,
-      "grad_norm": 0.6520631909370422,
-      "learning_rate": 5.891927083333334e-06,
-      "loss": 1.1181,
-      "mean_token_accuracy": 0.751795919239521,
-      "num_tokens": 4113582.0,
-      "step": 680
-    },
-    {
-      "entropy": 1.1519750133156776,
-      "epoch": 0.8712121212121212,
-      "grad_norm": 0.6247655153274536,
-      "learning_rate": 5.826822916666667e-06,
-      "loss": 1.1204,
-      "mean_token_accuracy": 0.7506368085741997,
-      "num_tokens": 4174983.0,
-      "step": 690
-    },
-    {
-      "entropy": 1.1486561581492425,
-      "epoch": 0.8838383838383839,
-      "grad_norm": 0.620272159576416,
-      "learning_rate": 5.761718750000001e-06,
-      "loss": 1.1191,
-      "mean_token_accuracy": 0.7536391675472259,
-      "num_tokens": 4234465.0,
-      "step": 700
-    },
-    {
-      "entropy": 1.1504923462867738,
-      "epoch": 0.8964646464646465,
-      "grad_norm": 0.6308649182319641,
-      "learning_rate": 5.6966145833333344e-06,
-      "loss": 1.1224,
-      "mean_token_accuracy": 0.750646598637104,
-      "num_tokens": 4295955.0,
-      "step": 710
-    },
-    {
-      "entropy": 1.1561898440122604,
-      "epoch": 0.9090909090909091,
-      "grad_norm": 0.6629899740219116,
-      "learning_rate": 5.6315104166666665e-06,
-      "loss": 1.1238,
-      "mean_token_accuracy": 0.7507557719945908,
-      "num_tokens": 4357171.0,
-      "step": 720
-    },
-    {
-      "entropy": 1.1413449853658677,
-      "epoch": 0.9217171717171717,
-      "grad_norm": 0.5972346067428589,
-      "learning_rate": 5.56640625e-06,
-      "loss": 1.1038,
-      "mean_token_accuracy": 0.7554104939103127,
-      "num_tokens": 4417636.0,
-      "step": 730
-    },
-    {
-      "entropy": 1.1356119453907012,
-      "epoch": 0.9343434343434344,
-      "grad_norm": 0.6356479525566101,
-      "learning_rate": 5.501302083333334e-06,
-      "loss": 1.1005,
-      "mean_token_accuracy": 0.7565629109740257,
-      "num_tokens": 4477294.0,
-      "step": 740
-    },
-    {
-      "entropy": 1.1633600294589996,
-      "epoch": 0.946969696969697,
-      "grad_norm": 0.6416464447975159,
-      "learning_rate": 5.436197916666667e-06,
-      "loss": 1.1225,
-      "mean_token_accuracy": 0.7515855401754379,
-      "num_tokens": 4537503.0,
-      "step": 750
-    },
-    {
-      "entropy": 1.1527763932943345,
-      "epoch": 0.9595959595959596,
-      "grad_norm": 0.6126084327697754,
-      "learning_rate": 5.3710937500000005e-06,
-      "loss": 1.1184,
-      "mean_token_accuracy": 0.7526160582900048,
-      "num_tokens": 4598778.0,
-      "step": 760
-    },
-    {
-      "entropy": 1.1397768080234527,
-      "epoch": 0.9722222222222222,
-      "grad_norm": 0.6359922289848328,
-      "learning_rate": 5.305989583333334e-06,
-      "loss": 1.1144,
-      "mean_token_accuracy": 0.7548302739858628,
-      "num_tokens": 4658978.0,
-      "step": 770
-    },
-    {
-      "entropy": 1.1569962561130525,
-      "epoch": 0.9848484848484849,
-      "grad_norm": 0.6260409951210022,
-      "learning_rate": 5.240885416666667e-06,
-      "loss": 1.1213,
-      "mean_token_accuracy": 0.7512885302305221,
-      "num_tokens": 4720500.0,
-      "step": 780
-    },
-    {
-      "entropy": 1.1509152203798294,
-      "epoch": 0.9974747474747475,
-      "grad_norm": 0.6293452978134155,
-      "learning_rate": 5.17578125e-06,
-      "loss": 1.1227,
-      "mean_token_accuracy": 0.7519564241170883,
-      "num_tokens": 4781612.0,
-      "step": 790
-    },
-    {
-      "entropy": 1.1399024561047555,
-      "epoch": 1.0101010101010102,
-      "grad_norm": 0.664761483669281,
-      "learning_rate": 5.110677083333334e-06,
-      "loss": 1.1034,
-      "mean_token_accuracy": 0.7526706486940384,
-      "num_tokens": 4841359.0,
-      "step": 800
-    },
-    {
-      "entropy": 1.120214229822159,
-      "epoch": 1.0227272727272727,
-      "grad_norm": 0.5934865474700928,
-      "learning_rate": 5.045572916666667e-06,
-      "loss": 1.0857,
-      "mean_token_accuracy": 0.7593594208359719,
-      "num_tokens": 4901016.0,
-      "step": 810
-    },
-    {
-      "entropy": 1.1430341199040412,
-      "epoch": 1.0353535353535352,
-      "grad_norm": 0.6040735840797424,
-      "learning_rate": 4.98046875e-06,
-      "loss": 1.1165,
-      "mean_token_accuracy": 0.7525988414883613,
-      "num_tokens": 4961646.0,
-      "step": 820
-    },
-    {
-      "entropy": 1.1245349109172822,
-      "epoch": 1.047979797979798,
-      "grad_norm": 0.6277610063552856,
-      "learning_rate": 4.915364583333333e-06,
-      "loss": 1.0851,
-      "mean_token_accuracy": 0.7577921718358993,
-      "num_tokens": 5022365.0,
-      "step": 830
-    },
-    {
-      "entropy": 1.1204675793647767,
-      "epoch": 1.0606060606060606,
-      "grad_norm": 0.6260582804679871,
-      "learning_rate": 4.850260416666667e-06,
-      "loss": 1.0813,
-      "mean_token_accuracy": 0.7580071151256561,
-      "num_tokens": 5081972.0,
-      "step": 840
-    },
-    {
-      "entropy": 1.1222782507538795,
-      "epoch": 1.0732323232323233,
-      "grad_norm": 0.6023226976394653,
-      "learning_rate": 4.785156250000001e-06,
-      "loss": 1.0922,
-      "mean_token_accuracy": 0.7566258609294891,
-      "num_tokens": 5142184.0,
-      "step": 850
-    },
-    {
-      "entropy": 1.1227335944771766,
-      "epoch": 1.0858585858585859,
-      "grad_norm": 0.6206791996955872,
-      "learning_rate": 4.7200520833333336e-06,
-      "loss": 1.0994,
-      "mean_token_accuracy": 0.7540625646710396,
-      "num_tokens": 5203020.0,
-      "step": 860
-    },
-    {
-      "entropy": 1.1352888554334641,
-      "epoch": 1.0984848484848484,
-      "grad_norm": 0.6301055550575256,
-      "learning_rate": 4.654947916666667e-06,
-      "loss": 1.0958,
-      "mean_token_accuracy": 0.7562039017677307,
-      "num_tokens": 5263291.0,
-      "step": 870
-    },
-    {
-      "entropy": 1.1120088309049607,
-      "epoch": 1.1111111111111112,
-      "grad_norm": 0.6210020780563354,
-      "learning_rate": 4.58984375e-06,
-      "loss": 1.0793,
-      "mean_token_accuracy": 0.7590557768940925,
-      "num_tokens": 5323972.0,
-      "step": 880
-    },
-    {
-      "entropy": 1.0996666207909584,
-      "epoch": 1.1237373737373737,
-      "grad_norm": 0.6332690715789795,
-      "learning_rate": 4.524739583333334e-06,
-      "loss": 1.0717,
-      "mean_token_accuracy": 0.7615471586585045,
-      "num_tokens": 5383911.0,
-      "step": 890
-    },
-    {
-      "entropy": 1.127107810974121,
-      "epoch": 1.1363636363636362,
-      "grad_norm": 0.6505516767501831,
-      "learning_rate": 4.459635416666668e-06,
-      "loss": 1.1027,
-      "mean_token_accuracy": 0.7562421515583992,
-      "num_tokens": 5445417.0,
-      "step": 900
-    },
-    {
-      "entropy": 1.129740473628044,
-      "epoch": 1.148989898989899,
-      "grad_norm": 0.6406158804893494,
-      "learning_rate": 4.3945312500000005e-06,
-      "loss": 1.0879,
-      "mean_token_accuracy": 0.7587148532271385,
-      "num_tokens": 5505455.0,
-      "step": 910
-    },
-    {
-      "entropy": 1.1167259424924851,
-      "epoch": 1.1616161616161615,
-      "grad_norm": 0.6297397613525391,
-      "learning_rate": 4.329427083333333e-06,
-      "loss": 1.0752,
-      "mean_token_accuracy": 0.7604142814874649,
-      "num_tokens": 5565311.0,
-      "step": 920
-    },
-    {
-      "entropy": 1.1037891641259194,
-      "epoch": 1.1742424242424243,
-      "grad_norm": 0.6490073204040527,
-      "learning_rate": 4.264322916666667e-06,
-      "loss": 1.0686,
-      "mean_token_accuracy": 0.7610052570700645,
-      "num_tokens": 5625358.0,
-      "step": 930
-    },
-    {
-      "entropy": 1.1061881184577942,
-      "epoch": 1.1868686868686869,
-      "grad_norm": 0.6366387009620667,
-      "learning_rate": 4.19921875e-06,
-      "loss": 1.0868,
-      "mean_token_accuracy": 0.7574937298893929,
-      "num_tokens": 5686421.0,
-      "step": 940
-    },
-    {
-      "entropy": 1.1124324068427085,
-      "epoch": 1.1994949494949494,
-      "grad_norm": 0.6556055545806885,
-      "learning_rate": 4.134114583333334e-06,
-      "loss": 1.0694,
-      "mean_token_accuracy": 0.7602224007248879,
-      "num_tokens": 5745891.0,
-      "step": 950
-    },
-    {
-      "entropy": 1.1175200879573821,
-      "epoch": 1.2121212121212122,
-      "grad_norm": 0.6404849886894226,
-      "learning_rate": 4.0690104166666675e-06,
-      "loss": 1.081,
-      "mean_token_accuracy": 0.7568994402885437,
-      "num_tokens": 5806078.0,
-      "step": 960
-    },
-    {
-      "entropy": 1.1186978340148925,
-      "epoch": 1.2247474747474747,
-      "grad_norm": 0.6227584481239319,
-      "learning_rate": 4.00390625e-06,
-      "loss": 1.0791,
-      "mean_token_accuracy": 0.759756401181221,
-      "num_tokens": 5866369.0,
-      "step": 970
-    },
-    {
-      "entropy": 1.122128139436245,
-      "epoch": 1.2373737373737375,
-      "grad_norm": 0.6616361141204834,
-      "learning_rate": 3.938802083333333e-06,
-      "loss": 1.0937,
-      "mean_token_accuracy": 0.7582718566060066,
-      "num_tokens": 5926217.0,
-      "step": 980
-    },
-    {
-      "entropy": 1.122861033678055,
-      "epoch": 1.25,
-      "grad_norm": 0.6384168267250061,
-      "learning_rate": 3.873697916666667e-06,
-      "loss": 1.0978,
-      "mean_token_accuracy": 0.7549964562058449,
-      "num_tokens": 5987666.0,
-      "step": 990
-    },
-    {
-      "entropy": 1.1277505576610565,
-      "epoch": 1.2626262626262625,
-      "grad_norm": 0.6038117408752441,
-      "learning_rate": 3.8085937500000002e-06,
-      "loss": 1.0952,
-      "mean_token_accuracy": 0.755272176861763,
-      "num_tokens": 6048708.0,
-      "step": 1000
-    },
-    {
-      "entropy": 1.1120157346129418,
-      "epoch": 1.2752525252525253,
-      "grad_norm": 0.6418159604072571,
-      "learning_rate": 3.7434895833333336e-06,
-      "loss": 1.078,
-      "mean_token_accuracy": 0.7594122514128685,
-      "num_tokens": 6109652.0,
-      "step": 1010
-    },
-    {
-      "entropy": 1.101425115764141,
-      "epoch": 1.2878787878787878,
-      "grad_norm": 0.6218425035476685,
-      "learning_rate": 3.6783854166666673e-06,
-      "loss": 1.0688,
-      "mean_token_accuracy": 0.7604865297675133,
-      "num_tokens": 6169125.0,
-      "step": 1020
-    },
-    {
-      "entropy": 1.1007713869214057,
-      "epoch": 1.3005050505050506,
-      "grad_norm": 0.6429149508476257,
-      "learning_rate": 3.61328125e-06,
-      "loss": 1.0581,
-      "mean_token_accuracy": 0.7621071562170982,
-      "num_tokens": 6230303.0,
-      "step": 1030
-    },
-    {
-      "entropy": 1.1094096556305886,
-      "epoch": 1.3131313131313131,
-      "grad_norm": 0.6489748358726501,
-      "learning_rate": 3.5481770833333335e-06,
-      "loss": 1.0715,
-      "mean_token_accuracy": 0.7599423810839653,
-      "num_tokens": 6291396.0,
-      "step": 1040
-    },
-    {
-      "entropy": 1.0827289715409278,
-      "epoch": 1.3257575757575757,
-      "grad_norm": 0.6485461592674255,
-      "learning_rate": 3.483072916666667e-06,
-      "loss": 1.0584,
-      "mean_token_accuracy": 0.7630694910883904,
-      "num_tokens": 6351579.0,
-      "step": 1050
-    },
-    {
-      "entropy": 1.114325873553753,
-      "epoch": 1.3383838383838385,
-      "grad_norm": 0.6261104941368103,
-      "learning_rate": 3.41796875e-06,
-      "loss": 1.0764,
-      "mean_token_accuracy": 0.7585488513112069,
-      "num_tokens": 6411662.0,
-      "step": 1060
-    },
-    {
-      "entropy": 1.1271554425358772,
-      "epoch": 1.351010101010101,
-      "grad_norm": 0.6522034406661987,
-      "learning_rate": 3.3528645833333334e-06,
-      "loss": 1.0902,
-      "mean_token_accuracy": 0.7562535598874092,
-      "num_tokens": 6473505.0,
-      "step": 1070
-    },
-    {
-      "entropy": 1.1013643085956573,
-      "epoch": 1.3636363636363638,
-      "grad_norm": 0.6176674962043762,
-      "learning_rate": 3.287760416666667e-06,
-      "loss": 1.065,
-      "mean_token_accuracy": 0.763075165450573,
-      "num_tokens": 6533580.0,
-      "step": 1080
-    },
-    {
-      "entropy": 1.098090337216854,
-      "epoch": 1.3762626262626263,
-      "grad_norm": 0.6540253758430481,
-      "learning_rate": 3.2226562500000004e-06,
-      "loss": 1.0596,
-      "mean_token_accuracy": 0.7616770043969154,
-      "num_tokens": 6593481.0,
-      "step": 1090
-    },
-    {
-      "entropy": 1.1176372200250626,
-      "epoch": 1.3888888888888888,
-      "grad_norm": 0.6754550933837891,
-      "learning_rate": 3.1575520833333333e-06,
-      "loss": 1.0861,
-      "mean_token_accuracy": 0.7573029339313507,
-      "num_tokens": 6653967.0,
-      "step": 1100
-    },
-    {
-      "entropy": 1.1040414482355119,
-      "epoch": 1.4015151515151514,
-      "grad_norm": 0.6022531986236572,
-      "learning_rate": 3.092447916666667e-06,
-      "loss": 1.0573,
-      "mean_token_accuracy": 0.7612267225980759,
-      "num_tokens": 6714685.0,
-      "step": 1110
-    },
-    {
-      "entropy": 1.0926991075277328,
-      "epoch": 1.4141414141414141,
-      "grad_norm": 0.6621010303497314,
-      "learning_rate": 3.0273437500000003e-06,
-      "loss": 1.0637,
-      "mean_token_accuracy": 0.7612977519631385,
-      "num_tokens": 6774659.0,
-      "step": 1120
-    },
-    {
-      "entropy": 1.1095534324645997,
-      "epoch": 1.4267676767676767,
-      "grad_norm": 0.62503981590271,
-      "learning_rate": 2.962239583333333e-06,
-      "loss": 1.0701,
-      "mean_token_accuracy": 0.7618053883314133,
-      "num_tokens": 6834579.0,
-      "step": 1130
-    },
-    {
-      "entropy": 1.117809349298477,
-      "epoch": 1.4393939393939394,
-      "grad_norm": 0.6527109742164612,
-      "learning_rate": 2.897135416666667e-06,
-      "loss": 1.0747,
-      "mean_token_accuracy": 0.759482853114605,
-      "num_tokens": 6894074.0,
-      "step": 1140
-    },
-    {
-      "entropy": 1.1005077749490737,
-      "epoch": 1.452020202020202,
-      "grad_norm": 0.6720954775810242,
-      "learning_rate": 2.8320312500000002e-06,
-      "loss": 1.0607,
-      "mean_token_accuracy": 0.7621246844530105,
-      "num_tokens": 6953870.0,
-      "step": 1150
-    },
-    {
-      "entropy": 1.1236482918262483,
-      "epoch": 1.4646464646464645,
-      "grad_norm": 0.658524215221405,
-      "learning_rate": 2.7669270833333335e-06,
-      "loss": 1.0884,
-      "mean_token_accuracy": 0.7560836613178253,
-      "num_tokens": 7014553.0,
-      "step": 1160
-    },
-    {
-      "entropy": 1.1116504594683647,
-      "epoch": 1.4772727272727273,
-      "grad_norm": 0.6261802911758423,
-      "learning_rate": 2.7018229166666673e-06,
-      "loss": 1.0659,
-      "mean_token_accuracy": 0.7597616642713547,
-      "num_tokens": 7076291.0,
-      "step": 1170
-    },
-    {
-      "entropy": 1.073892480134964,
-      "epoch": 1.4898989898989898,
-      "grad_norm": 0.6310375332832336,
-      "learning_rate": 2.63671875e-06,
-      "loss": 1.0524,
-      "mean_token_accuracy": 0.7628733053803444,
-      "num_tokens": 7137305.0,
-      "step": 1180
-    },
-    {
-      "entropy": 1.0975843235850333,
-      "epoch": 1.5025252525252526,
-      "grad_norm": 0.638482391834259,
-      "learning_rate": 2.5716145833333334e-06,
-      "loss": 1.0679,
-      "mean_token_accuracy": 0.7603248566389084,
-      "num_tokens": 7198239.0,
-      "step": 1190
-    },
-    {
-      "entropy": 1.0986508697271347,
-      "epoch": 1.5151515151515151,
-      "grad_norm": 0.640065610408783,
-      "learning_rate": 2.506510416666667e-06,
-      "loss": 1.0666,
-      "mean_token_accuracy": 0.7622530281543731,
-      "num_tokens": 7257847.0,
-      "step": 1200
-    }
-  ],
-  "logging_steps": 10,
-  "max_steps": 1584,
-  "num_input_tokens_seen": 0,
-  "num_train_epochs": 2,
-  "save_steps": 200,
-  "stateful_callbacks": {
-    "TrainerControl": {
-      "args": {
-        "should_epoch_stop": false,
-        "should_evaluate": false,
-        "should_log": false,
-        "should_save": true,
-        "should_training_stop": false
-      },
-      "attributes": {}
-    }
-  },
-  "total_flos": 4.110405675434312e+17,
-  "train_batch_size": 8,
-  "trial_name": null,
-  "trial_params": null
-}

checkpoint-1200/training_args.bin

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:8eeb71c14deb91ac5fd11522db45cb3275c9164415fcbefc9d00cac27a27f0a3
-size 6417