Add checkpoint-800

.gitattributes

@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 checkpoints/checkpoint-200/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoints/checkpoint-400/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoints/checkpoint-600/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoints/checkpoint-800/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoints/checkpoint-800/README.md

@@ -0,0 +1,210 @@
+---
+base_model: OpenPathAI/Orbit-3-Nano
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:OpenPathAI/Orbit-3-Nano
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

checkpoints/checkpoint-800/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Lfm2ForCausalLM",
+    "parent_library": "transformers.models.lfm2.modeling_lfm2",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "OpenPathAI/Orbit-3-Nano",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:language|text).*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?\\.(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj))|(?:\\bmodel\\.layers\\.[\\d]{1,}(?!.*\\.experts\\.\\d)\\..*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?\\.(?:(?:q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

checkpoints/checkpoint-800/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da1003f81d13a8a8e53d6ea09d2c5670e09d572499a335220886b42deda9a0c9
+size 4724960

checkpoints/checkpoint-800/chat_template.jinja

@@ -0,0 +1,125 @@
+{{- bos_token -}}
+{%- set preserve_thinking = preserve_thinking | default(false) -%}
+
+{%- macro format_arg_value(arg_value) -%}
+    {%- if arg_value is string -%}
+        {{- "'" + (arg_value | replace("\\", "\\\\") | replace("'", "\\'") | replace("\n", "\\n") | replace("\r", "\\r")) + "'" -}}
+    {%- elif arg_value is mapping or arg_value is iterable -%}
+        {{- arg_value | tojson -}}
+    {%- else -%}
+        {{- arg_value | string -}}
+    {%- endif -%}
+{%- endmacro -%}
+
+{%- macro parse_content(content) -%}
+    {%- if content is string -%}
+        {{- content -}}
+    {%- elif content is mapping -%}
+        {{- content | tojson -}}
+    {%- elif content is iterable -%}
+        {%- set _ns = namespace(result="") -%}
+        {%- for item in content -%}
+            {%- if item is string -%}
+                {%- set _ns.result = _ns.result + item -%}
+            {%- elif item is mapping and item.get("type") == "image" -%}
+                {%- set _ns.result = _ns.result + "<image>" -%}
+            {%- elif item is mapping and item.get("type") == "text" -%}
+                {%- set _ns.result = _ns.result + ((item.get("text") or "") | string) -%}
+            {%- else -%}
+                {%- set _ns.result = _ns.result + (item | tojson) -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {{- _ns.result -}}
+    {%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_tool_calls(tool_calls) -%}
+    {%- set tool_calls_ns = namespace(tool_calls=[]) -%}
+    {%- for tool_call in tool_calls -%}
+        {%- set func = tool_call["function"] if "function" in tool_call else tool_call -%}
+        {%- set func_name = func["name"] -%}
+        {%- set func_args = func.get("arguments") -%}
+        {%- set args_ns = namespace(arg_strings=[]) -%}
+        {%- if func_args is mapping -%}
+            {%- for arg_name, arg_value in func_args.items() -%}
+                {%- set args_ns.arg_strings = args_ns.arg_strings + [arg_name + "=" + format_arg_value(arg_value)] -%}
+            {%- endfor -%}
+        {%- elif func_args is string and (func_args | trim) not in ["", "{}", "null"] -%}
+            {{- raise_exception("Tool call arguments must be a mapping, got a JSON-encoded string: parse arguments with json.loads() before applying the chat template") -}}
+        {%- endif -%}
+        {%- set tool_calls_ns.tool_calls = tool_calls_ns.tool_calls + [func_name + "(" + (args_ns.arg_strings | join(", ")) + ")"] -%}
+    {%- endfor -%}
+    {{- "<|tool_call_start|>[" + (tool_calls_ns.tool_calls | join(", ")) + "]<|tool_call_end|>" -}}
+{%- endmacro -%}
+
+{%- set ns = namespace(system_prompt="", last_user_index=-1) -%}
+{%- if messages and messages[0]["role"] == "system" -%}
+    {%- if messages[0].get("content") -%}
+        {%- set ns.system_prompt = parse_content(messages[0]["content"]) -%}
+    {%- endif -%}
+    {%- set messages = messages[1:] -%}
+{%- endif -%}
+{%- if tools -%}
+    {%- set ns.system_prompt = ns.system_prompt + ("\n" if ns.system_prompt else "") + "List of tools: [" -%}
+    {%- for tool in tools -%}
+        {%- if tool is not string -%}
+            {%- set tool = tool | tojson -%}
+        {%- endif -%}
+        {%- set ns.system_prompt = ns.system_prompt + tool -%}
+        {%- if not loop.last -%}
+            {%- set ns.system_prompt = ns.system_prompt + ", " -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns.system_prompt = ns.system_prompt + "]" -%}
+{%- endif -%}
+{%- if ns.system_prompt -%}
+    {{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
+{%- endif -%}
+{%- for message in messages -%}
+    {%- if message["role"] == "user" -%}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif -%}
+{%- endfor -%}
+{%- for message in messages -%}
+    {{- "<|im_start|>" + message.role + "\n" -}}
+    {%- if message.role == "assistant" -%}
+        {%- generation -%}
+        {%- set keep_thinking = preserve_thinking or loop.index0 > ns.last_user_index -%}
+        {%- set thinking = message.thinking or message.reasoning or message.reasoning_content -%}
+        {%- set thinking = thinking if thinking is string else "" -%}
+        {%- if thinking and keep_thinking -%}
+            {{- "<think>" + thinking + "</think>" -}}
+        {%- endif -%}
+        {%- set _cfm_tag = "CONTINUE_FINAL_MESSAGE_TAG " -%}
+        {%- set _has_cfm = false -%}
+        {%- set content = "" -%}
+        {%- if message.get("content") -%}
+            {%- set content = parse_content(message.content) -%}
+        {%- endif -%}
+        {%- if not keep_thinking and "</think>" in content -%}
+            {%- set content = content.split("</think>")[-1] | trim -%}
+        {%- endif -%}
+        {%- if content.endswith(_cfm_tag) -%}
+            {%- set _has_cfm = true -%}
+            {%- set _trunc_len = (content | length) - (_cfm_tag | length) -%}
+            {%- set content = content[:_trunc_len] -%}
+        {%- endif -%}
+        {{- content -}}
+        {%- if message.tool_calls -%}
+            {{- render_tool_calls(message.tool_calls) -}}
+        {%- endif -%}
+        {%- if _has_cfm -%}
+            {{- _cfm_tag -}}
+        {%- endif -%}
+        {{- "<|im_end|>\n" -}}
+        {%- endgeneration -%}
+    {%- else %}
+        {%- if message.get("content") -%}
+            {{- parse_content(message["content"]) -}}
+        {%- endif -%}
+        {{- "<|im_end|>\n" -}}
+    {%- endif %}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{- "<|im_start|>assistant\n<think>" -}}
+{%- endif -%}

checkpoints/checkpoint-800/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e566bd39b84343606ae29ebdf27de4ef9c9eb961587ec2115fece20ba9b65514
+size 2455755

checkpoints/checkpoint-800/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b79e62ad18fda7454cc55501bec8eb593d22e440e078eb7e7dbab5e57f78b1d
+size 14645

checkpoints/checkpoint-800/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:168a8546b1b165890115e89e6876a340813ee6c3a442c5bee90ef296f81e0437
+size 1383

checkpoints/checkpoint-800/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:caa0b20ecbf060640ea51474e6fb11f68631dfe2428cb5e9a65f28a00b2697c1
+size 1465

checkpoints/checkpoint-800/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41
+size 17905598

checkpoints/checkpoint-800/tokenizer_config.json

@@ -0,0 +1,1008 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "is_local": false,
+  "legacy": false,
+  "local_files_only": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|pad|>",
+  "padding_side": "right",
+  "tokenizer_class": "TokenizersBackend",
+  "use_default_system_prompt": false,
+  "added_tokens_decoder": {
+    "124893": {
+      "content": "<|pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124894": {
+      "content": "<|startoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124895": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124896": {
+      "content": "<|fim_pre|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124897": {
+      "content": "<|fim_mid|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124898": {
+      "content": "<|fim_suf|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124899": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124900": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124901": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "124902": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "124903": {
+      "content": "<|tool_list_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124904": {
+      "content": "<|tool_list_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124905": {
+      "content": "<|tool_call_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "124906": {
+      "content": "<|tool_call_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "124907": {
+      "content": "<image>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124908": {
+      "content": "<|img_row_1_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124909": {
+      "content": "<|img_row_1_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124910": {
+      "content": "<|img_row_1_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124911": {
+      "content": "<|img_row_1_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124912": {
+      "content": "<|img_row_1_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124913": {
+      "content": "<|img_row_1_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124914": {
+      "content": "<|img_row_1_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124915": {
+      "content": "<|img_row_1_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124916": {
+      "content": "<|img_row_1_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124917": {
+      "content": "<|img_row_1_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124918": {
+      "content": "<|img_row_2_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124919": {
+      "content": "<|img_row_2_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124920": {
+      "content": "<|img_row_2_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124921": {
+      "content": "<|img_row_2_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124922": {
+      "content": "<|img_row_2_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124923": {
+      "content": "<|img_row_2_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124924": {
+      "content": "<|img_row_2_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124925": {
+      "content": "<|img_row_2_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124926": {
+      "content": "<|img_row_2_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124927": {
+      "content": "<|img_row_2_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124928": {
+      "content": "<|img_row_3_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124929": {
+      "content": "<|img_row_3_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124930": {
+      "content": "<|img_row_3_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124931": {
+      "content": "<|img_row_3_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124932": {
+      "content": "<|img_row_3_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124933": {
+      "content": "<|img_row_3_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124934": {
+      "content": "<|img_row_3_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124935": {
+      "content": "<|img_row_3_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124936": {
+      "content": "<|img_row_3_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124937": {
+      "content": "<|img_row_3_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124938": {
+      "content": "<|img_row_4_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124939": {
+      "content": "<|img_row_4_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124940": {
+      "content": "<|img_row_4_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124941": {
+      "content": "<|img_row_4_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124942": {
+      "content": "<|img_row_4_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124943": {
+      "content": "<|img_row_4_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124944": {
+      "content": "<|img_row_4_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124945": {
+      "content": "<|img_row_4_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124946": {
+      "content": "<|img_row_4_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124947": {
+      "content": "<|img_row_4_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124948": {
+      "content": "<|img_row_5_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124949": {
+      "content": "<|img_row_5_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124950": {
+      "content": "<|img_row_5_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124951": {
+      "content": "<|img_row_5_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124952": {
+      "content": "<|img_row_5_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124953": {
+      "content": "<|img_row_5_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124954": {
+      "content": "<|img_row_5_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124955": {
+      "content": "<|img_row_5_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124956": {
+      "content": "<|img_row_5_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124957": {
+      "content": "<|img_row_5_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124958": {
+      "content": "<|img_row_6_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124959": {
+      "content": "<|img_row_6_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124960": {
+      "content": "<|img_row_6_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124961": {
+      "content": "<|img_row_6_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124962": {
+      "content": "<|img_row_6_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124963": {
+      "content": "<|img_row_6_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124964": {
+      "content": "<|img_row_6_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124965": {
+      "content": "<|img_row_6_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124966": {
+      "content": "<|img_row_6_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124967": {
+      "content": "<|img_row_6_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124968": {
+      "content": "<|img_row_7_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124969": {
+      "content": "<|img_row_7_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124970": {
+      "content": "<|img_row_7_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124971": {
+      "content": "<|img_row_7_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124972": {
+      "content": "<|img_row_7_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124973": {
+      "content": "<|img_row_7_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124974": {
+      "content": "<|img_row_7_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124975": {
+      "content": "<|img_row_7_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124976": {
+      "content": "<|img_row_7_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124977": {
+      "content": "<|img_row_7_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124978": {
+      "content": "<|img_row_8_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124979": {
+      "content": "<|img_row_8_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124980": {
+      "content": "<|img_row_8_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124981": {
+      "content": "<|img_row_8_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124982": {
+      "content": "<|img_row_8_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124983": {
+      "content": "<|img_row_8_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124984": {
+      "content": "<|img_row_8_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124985": {
+      "content": "<|img_row_8_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124986": {
+      "content": "<|img_row_8_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124987": {
+      "content": "<|img_row_8_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124988": {
+      "content": "<|img_row_9_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124989": {
+      "content": "<|img_row_9_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124990": {
+      "content": "<|img_row_9_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124991": {
+      "content": "<|img_row_9_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124992": {
+      "content": "<|img_row_9_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124993": {
+      "content": "<|img_row_9_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124994": {
+      "content": "<|img_row_9_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124995": {
+      "content": "<|img_row_9_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124996": {
+      "content": "<|img_row_9_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124997": {
+      "content": "<|img_row_9_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124998": {
+      "content": "<|img_row_10_col_1|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "124999": {
+      "content": "<|img_row_10_col_2|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125000": {
+      "content": "<|img_row_10_col_3|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125001": {
+      "content": "<|img_row_10_col_4|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125002": {
+      "content": "<|img_row_10_col_5|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125003": {
+      "content": "<|img_row_10_col_6|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125004": {
+      "content": "<|img_row_10_col_7|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125005": {
+      "content": "<|img_row_10_col_8|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125006": {
+      "content": "<|img_row_10_col_9|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125007": {
+      "content": "<|img_row_10_col_10|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125008": {
+      "content": "<|img_thumbnail|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125009": {
+      "content": "<|image_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125010": {
+      "content": "<|image_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125011": {
+      "content": "<|image_split|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125012": {
+      "content": "<|audio_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125013": {
+      "content": "<|text_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125014": {
+      "content": "<|text_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125015": {
+      "content": "<|mixed_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "125016": {
+      "content": "<|mixed_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

checkpoints/checkpoint-800/trainer_state.json

@@ -0,0 +1,594 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.3175863437872172,
+  "eval_steps": 500,
+  "global_step": 800,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.003969829297340214,
+      "grad_norm": 0.6961175799369812,
+      "learning_rate": 0.00016,
+      "loss": 3.2075271606445312,
+      "step": 10
+    },
+    {
+      "epoch": 0.007939658594680429,
+      "grad_norm": 0.9305199384689331,
+      "learning_rate": 0.00019978799522989268,
+      "loss": 3.1885648727416993,
+      "step": 20
+    },
+    {
+      "epoch": 0.011909487892020643,
+      "grad_norm": 0.8452768325805664,
+      "learning_rate": 0.00019952298926725852,
+      "loss": 2.9959283828735352,
+      "step": 30
+    },
+    {
+      "epoch": 0.015879317189360857,
+      "grad_norm": 0.9384715557098389,
+      "learning_rate": 0.00019925798330462438,
+      "loss": 3.132890510559082,
+      "step": 40
+    },
+    {
+      "epoch": 0.019849146486701073,
+      "grad_norm": 0.9872943162918091,
+      "learning_rate": 0.0001989929773419902,
+      "loss": 3.1314294815063475,
+      "step": 50
+    },
+    {
+      "epoch": 0.023818975784041286,
+      "grad_norm": 0.8298355937004089,
+      "learning_rate": 0.00019872797137935604,
+      "loss": 3.24124755859375,
+      "step": 60
+    },
+    {
+      "epoch": 0.027788805081381502,
+      "grad_norm": 0.8308970928192139,
+      "learning_rate": 0.00019846296541672188,
+      "loss": 3.0996198654174805,
+      "step": 70
+    },
+    {
+      "epoch": 0.031758634378721715,
+      "grad_norm": 0.8442180156707764,
+      "learning_rate": 0.0001981979594540877,
+      "loss": 3.0475082397460938,
+      "step": 80
+    },
+    {
+      "epoch": 0.03572846367606193,
+      "grad_norm": 0.9131914377212524,
+      "learning_rate": 0.00019793295349145357,
+      "loss": 3.1756013870239257,
+      "step": 90
+    },
+    {
+      "epoch": 0.03969829297340215,
+      "grad_norm": 0.9973925352096558,
+      "learning_rate": 0.0001976679475288194,
+      "loss": 3.159317398071289,
+      "step": 100
+    },
+    {
+      "epoch": 0.043668122270742356,
+      "grad_norm": 0.9906535148620605,
+      "learning_rate": 0.00019740294156618524,
+      "loss": 3.086680603027344,
+      "step": 110
+    },
+    {
+      "epoch": 0.04763795156808257,
+      "grad_norm": 0.8675328493118286,
+      "learning_rate": 0.0001971379356035511,
+      "loss": 3.0809303283691407,
+      "step": 120
+    },
+    {
+      "epoch": 0.05160778086542279,
+      "grad_norm": 1.2756576538085938,
+      "learning_rate": 0.00019687292964091694,
+      "loss": 2.965633010864258,
+      "step": 130
+    },
+    {
+      "epoch": 0.055577610162763004,
+      "grad_norm": 0.8540228605270386,
+      "learning_rate": 0.00019660792367828277,
+      "loss": 3.11167049407959,
+      "step": 140
+    },
+    {
+      "epoch": 0.05954743946010321,
+      "grad_norm": 0.8471305966377258,
+      "learning_rate": 0.0001963429177156486,
+      "loss": 2.9493082046508787,
+      "step": 150
+    },
+    {
+      "epoch": 0.06351726875744343,
+      "grad_norm": 1.0690556764602661,
+      "learning_rate": 0.00019607791175301444,
+      "loss": 3.1130550384521483,
+      "step": 160
+    },
+    {
+      "epoch": 0.06748709805478365,
+      "grad_norm": 0.9682918787002563,
+      "learning_rate": 0.0001958129057903803,
+      "loss": 3.041396713256836,
+      "step": 170
+    },
+    {
+      "epoch": 0.07145692735212386,
+      "grad_norm": 0.9328762292861938,
+      "learning_rate": 0.00019554789982774613,
+      "loss": 3.1088594436645507,
+      "step": 180
+    },
+    {
+      "epoch": 0.07542675664946408,
+      "grad_norm": 0.981307327747345,
+      "learning_rate": 0.000195282893865112,
+      "loss": 3.1146469116210938,
+      "step": 190
+    },
+    {
+      "epoch": 0.0793965859468043,
+      "grad_norm": 1.0752034187316895,
+      "learning_rate": 0.00019501788790247783,
+      "loss": 3.0602563858032226,
+      "step": 200
+    },
+    {
+      "epoch": 0.0833664152441445,
+      "grad_norm": 0.8543508052825928,
+      "learning_rate": 0.00019475288193984363,
+      "loss": 2.989186477661133,
+      "step": 210
+    },
+    {
+      "epoch": 0.08733624454148471,
+      "grad_norm": 0.9234327077865601,
+      "learning_rate": 0.0001944878759772095,
+      "loss": 3.0583696365356445,
+      "step": 220
+    },
+    {
+      "epoch": 0.09130607383882493,
+      "grad_norm": 0.8820871710777283,
+      "learning_rate": 0.00019422287001457533,
+      "loss": 2.868353271484375,
+      "step": 230
+    },
+    {
+      "epoch": 0.09527590313616514,
+      "grad_norm": 0.9947519898414612,
+      "learning_rate": 0.0001939578640519412,
+      "loss": 3.110881042480469,
+      "step": 240
+    },
+    {
+      "epoch": 0.09924573243350536,
+      "grad_norm": 0.8357694745063782,
+      "learning_rate": 0.00019369285808930702,
+      "loss": 3.01332950592041,
+      "step": 250
+    },
+    {
+      "epoch": 0.10321556173084558,
+      "grad_norm": 0.9128127694129944,
+      "learning_rate": 0.00019342785212667288,
+      "loss": 3.149305725097656,
+      "step": 260
+    },
+    {
+      "epoch": 0.10718539102818579,
+      "grad_norm": 0.9448281526565552,
+      "learning_rate": 0.0001931628461640387,
+      "loss": 3.172308921813965,
+      "step": 270
+    },
+    {
+      "epoch": 0.11115522032552601,
+      "grad_norm": 1.0875459909439087,
+      "learning_rate": 0.00019289784020140452,
+      "loss": 3.1815589904785155,
+      "step": 280
+    },
+    {
+      "epoch": 0.11512504962286621,
+      "grad_norm": 0.9254504442214966,
+      "learning_rate": 0.00019263283423877038,
+      "loss": 3.1059280395507813,
+      "step": 290
+    },
+    {
+      "epoch": 0.11909487892020643,
+      "grad_norm": 0.9273567795753479,
+      "learning_rate": 0.00019236782827613622,
+      "loss": 3.063690757751465,
+      "step": 300
+    },
+    {
+      "epoch": 0.12306470821754664,
+      "grad_norm": 0.9300341010093689,
+      "learning_rate": 0.00019210282231350208,
+      "loss": 2.989412307739258,
+      "step": 310
+    },
+    {
+      "epoch": 0.12703453751488686,
+      "grad_norm": 1.092647910118103,
+      "learning_rate": 0.0001918378163508679,
+      "loss": 2.9342714309692384,
+      "step": 320
+    },
+    {
+      "epoch": 0.13100436681222707,
+      "grad_norm": 0.7696720361709595,
+      "learning_rate": 0.00019157281038823375,
+      "loss": 2.966893768310547,
+      "step": 330
+    },
+    {
+      "epoch": 0.1349741961095673,
+      "grad_norm": 0.9847505688667297,
+      "learning_rate": 0.00019130780442559958,
+      "loss": 3.054021644592285,
+      "step": 340
+    },
+    {
+      "epoch": 0.1389440254069075,
+      "grad_norm": 1.0354719161987305,
+      "learning_rate": 0.00019104279846296541,
+      "loss": 2.9900562286376955,
+      "step": 350
+    },
+    {
+      "epoch": 0.14291385470424772,
+      "grad_norm": 0.8888474106788635,
+      "learning_rate": 0.00019077779250033128,
+      "loss": 3.1933818817138673,
+      "step": 360
+    },
+    {
+      "epoch": 0.14688368400158794,
+      "grad_norm": 1.0046141147613525,
+      "learning_rate": 0.0001905127865376971,
+      "loss": 2.920476722717285,
+      "step": 370
+    },
+    {
+      "epoch": 0.15085351329892815,
+      "grad_norm": 1.039939522743225,
+      "learning_rate": 0.00019024778057506294,
+      "loss": 3.1360284805297853,
+      "step": 380
+    },
+    {
+      "epoch": 0.15482334259626837,
+      "grad_norm": 0.9551829099655151,
+      "learning_rate": 0.0001899827746124288,
+      "loss": 3.144388198852539,
+      "step": 390
+    },
+    {
+      "epoch": 0.1587931718936086,
+      "grad_norm": 0.9663012027740479,
+      "learning_rate": 0.00018971776864979464,
+      "loss": 2.9875814437866213,
+      "step": 400
+    },
+    {
+      "epoch": 0.1627630011909488,
+      "grad_norm": 0.9962781071662903,
+      "learning_rate": 0.00018945276268716047,
+      "loss": 3.0062061309814454,
+      "step": 410
+    },
+    {
+      "epoch": 0.166732830488289,
+      "grad_norm": 1.001230001449585,
+      "learning_rate": 0.0001891877567245263,
+      "loss": 3.0834712982177734,
+      "step": 420
+    },
+    {
+      "epoch": 0.1707026597856292,
+      "grad_norm": 0.9189774990081787,
+      "learning_rate": 0.00018892275076189214,
+      "loss": 2.885240936279297,
+      "step": 430
+    },
+    {
+      "epoch": 0.17467248908296942,
+      "grad_norm": 1.0268958806991577,
+      "learning_rate": 0.000188657744799258,
+      "loss": 3.1581113815307615,
+      "step": 440
+    },
+    {
+      "epoch": 0.17864231838030964,
+      "grad_norm": 0.9994209408760071,
+      "learning_rate": 0.00018839273883662383,
+      "loss": 2.8793590545654295,
+      "step": 450
+    },
+    {
+      "epoch": 0.18261214767764986,
+      "grad_norm": 1.0613411664962769,
+      "learning_rate": 0.0001881277328739897,
+      "loss": 3.029997444152832,
+      "step": 460
+    },
+    {
+      "epoch": 0.18658197697499007,
+      "grad_norm": 1.0501508712768555,
+      "learning_rate": 0.0001878627269113555,
+      "loss": 3.150674819946289,
+      "step": 470
+    },
+    {
+      "epoch": 0.1905518062723303,
+      "grad_norm": 1.0706281661987305,
+      "learning_rate": 0.00018759772094872134,
+      "loss": 3.0236751556396486,
+      "step": 480
+    },
+    {
+      "epoch": 0.1945216355696705,
+      "grad_norm": 0.856622040271759,
+      "learning_rate": 0.0001873327149860872,
+      "loss": 2.9095502853393556,
+      "step": 490
+    },
+    {
+      "epoch": 0.19849146486701072,
+      "grad_norm": 0.9528838396072388,
+      "learning_rate": 0.00018706770902345303,
+      "loss": 3.001302146911621,
+      "step": 500
+    },
+    {
+      "epoch": 0.20246129416435094,
+      "grad_norm": 1.0344288349151611,
+      "learning_rate": 0.0001868027030608189,
+      "loss": 3.106768035888672,
+      "step": 510
+    },
+    {
+      "epoch": 0.20643112346169115,
+      "grad_norm": 0.9366422891616821,
+      "learning_rate": 0.00018653769709818473,
+      "loss": 2.99255428314209,
+      "step": 520
+    },
+    {
+      "epoch": 0.21040095275903137,
+      "grad_norm": 0.9982316493988037,
+      "learning_rate": 0.00018627269113555056,
+      "loss": 2.904929351806641,
+      "step": 530
+    },
+    {
+      "epoch": 0.21437078205637158,
+      "grad_norm": 1.0310651063919067,
+      "learning_rate": 0.0001860076851729164,
+      "loss": 3.076841926574707,
+      "step": 540
+    },
+    {
+      "epoch": 0.2183406113537118,
+      "grad_norm": 1.015830397605896,
+      "learning_rate": 0.00018574267921028223,
+      "loss": 2.995553398132324,
+      "step": 550
+    },
+    {
+      "epoch": 0.22231044065105202,
+      "grad_norm": 0.9116207957267761,
+      "learning_rate": 0.0001854776732476481,
+      "loss": 3.0907325744628906,
+      "step": 560
+    },
+    {
+      "epoch": 0.22628026994839223,
+      "grad_norm": 0.983268141746521,
+      "learning_rate": 0.00018521266728501392,
+      "loss": 2.9128732681274414,
+      "step": 570
+    },
+    {
+      "epoch": 0.23025009924573242,
+      "grad_norm": 1.0083118677139282,
+      "learning_rate": 0.00018494766132237978,
+      "loss": 3.06164665222168,
+      "step": 580
+    },
+    {
+      "epoch": 0.23421992854307264,
+      "grad_norm": 0.9371793270111084,
+      "learning_rate": 0.00018468265535974562,
+      "loss": 3.110069465637207,
+      "step": 590
+    },
+    {
+      "epoch": 0.23818975784041285,
+      "grad_norm": 0.8907871842384338,
+      "learning_rate": 0.00018441764939711142,
+      "loss": 2.9918779373168944,
+      "step": 600
+    },
+    {
+      "epoch": 0.24215958713775307,
+      "grad_norm": 1.2298108339309692,
+      "learning_rate": 0.00018415264343447728,
+      "loss": 2.995768737792969,
+      "step": 610
+    },
+    {
+      "epoch": 0.24612941643509328,
+      "grad_norm": 1.1198155879974365,
+      "learning_rate": 0.00018388763747184312,
+      "loss": 2.905741500854492,
+      "step": 620
+    },
+    {
+      "epoch": 0.25009924573243353,
+      "grad_norm": 0.9270941019058228,
+      "learning_rate": 0.00018362263150920898,
+      "loss": 3.0602294921875,
+      "step": 630
+    },
+    {
+      "epoch": 0.2540690750297737,
+      "grad_norm": 0.907201886177063,
+      "learning_rate": 0.0001833576255465748,
+      "loss": 3.0065471649169924,
+      "step": 640
+    },
+    {
+      "epoch": 0.25803890432711396,
+      "grad_norm": 1.0519545078277588,
+      "learning_rate": 0.00018309261958394065,
+      "loss": 3.029597282409668,
+      "step": 650
+    },
+    {
+      "epoch": 0.26200873362445415,
+      "grad_norm": 0.8725332617759705,
+      "learning_rate": 0.00018282761362130648,
+      "loss": 2.8936578750610353,
+      "step": 660
+    },
+    {
+      "epoch": 0.26597856292179434,
+      "grad_norm": 1.040444254875183,
+      "learning_rate": 0.0001825626076586723,
+      "loss": 3.081193542480469,
+      "step": 670
+    },
+    {
+      "epoch": 0.2699483922191346,
+      "grad_norm": 0.9259696006774902,
+      "learning_rate": 0.00018229760169603817,
+      "loss": 2.9740297317504885,
+      "step": 680
+    },
+    {
+      "epoch": 0.27391822151647477,
+      "grad_norm": 1.0812491178512573,
+      "learning_rate": 0.000182032595733404,
+      "loss": 3.0157501220703127,
+      "step": 690
+    },
+    {
+      "epoch": 0.277888050813815,
+      "grad_norm": 1.0642049312591553,
+      "learning_rate": 0.00018176758977076984,
+      "loss": 2.996814727783203,
+      "step": 700
+    },
+    {
+      "epoch": 0.2818578801111552,
+      "grad_norm": 1.0878177881240845,
+      "learning_rate": 0.0001815025838081357,
+      "loss": 3.0396562576293946,
+      "step": 710
+    },
+    {
+      "epoch": 0.28582770940849545,
+      "grad_norm": 0.8735718131065369,
+      "learning_rate": 0.00018123757784550154,
+      "loss": 2.8770545959472655,
+      "step": 720
+    },
+    {
+      "epoch": 0.28979753870583563,
+      "grad_norm": 0.9658878445625305,
+      "learning_rate": 0.00018097257188286737,
+      "loss": 3.050862121582031,
+      "step": 730
+    },
+    {
+      "epoch": 0.2937673680031759,
+      "grad_norm": 0.958960771560669,
+      "learning_rate": 0.0001807075659202332,
+      "loss": 3.0131216049194336,
+      "step": 740
+    },
+    {
+      "epoch": 0.29773719730051607,
+      "grad_norm": 1.0989668369293213,
+      "learning_rate": 0.00018044255995759904,
+      "loss": 3.0553775787353517,
+      "step": 750
+    },
+    {
+      "epoch": 0.3017070265978563,
+      "grad_norm": 0.9090871214866638,
+      "learning_rate": 0.0001801775539949649,
+      "loss": 2.9768255233764647,
+      "step": 760
+    },
+    {
+      "epoch": 0.3056768558951965,
+      "grad_norm": 0.9842050671577454,
+      "learning_rate": 0.00017991254803233073,
+      "loss": 3.028487205505371,
+      "step": 770
+    },
+    {
+      "epoch": 0.30964668519253674,
+      "grad_norm": 1.1661285161972046,
+      "learning_rate": 0.0001796475420696966,
+      "loss": 2.9407962799072265,
+      "step": 780
+    },
+    {
+      "epoch": 0.31361651448987693,
+      "grad_norm": 1.0115314722061157,
+      "learning_rate": 0.00017938253610706243,
+      "loss": 3.02266845703125,
+      "step": 790
+    },
+    {
+      "epoch": 0.3175863437872172,
+      "grad_norm": 1.1729583740234375,
+      "learning_rate": 0.00017911753014442823,
+      "loss": 2.9814245223999025,
+      "step": 800
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 7557,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.547644300288e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoints/checkpoint-800/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae591a3a6737084c7522dfc1a295aa9ab6a59d7dface3e3999a058ffc77e97fc
+size 5841