diff --git a/.gitattributes b/.gitattributes
index a6344aac8c09253b3b630fb776ae94478aa0275b..e482705e4ae5c23839d78b80f0b31f60c1563406 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+best/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-100/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-200/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-300/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-400/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-544/tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/README.md b/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d1af9f5b761b92fa9895805dbdbf5622eef09280
--- /dev/null
+++ b/README.md
@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: transformers
+model_name: andy-D1-andyft
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
+---
+
+# Model Card for andy-D1-andyft
+
+This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+ 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 1.6.0
+- Transformers: 5.12.1
+- Pytorch: 2.4.0+cu121
+- Datasets: 5.0.0
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```
\ No newline at end of file
diff --git a/best/README.md b/best/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..dd49221a54a197d9f0cab86c7982e2b2fe0776b3
--- /dev/null
+++ b/best/README.md
@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/best/adapter_config.json b/best/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..07d800af2efdcfa371d6bb94e3b2ed9781fce6cc
--- /dev/null
+++ b/best/adapter_config.json
@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/best/adapter_model.safetensors b/best/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..5d0e36569b8174afb25790039533c2e02ed57e4a
--- /dev/null
+++ b/best/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bff2439b66b331f838870df1067c83cb040698f63f765305181cc00f7dc475cc
+size 161533192
diff --git a/best/chat_template.jinja b/best/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/best/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/best/tokenizer.json b/best/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..34510ff0037cd50428af467a17ead5a96140a32c
--- /dev/null
+++ b/best/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
diff --git a/best/tokenizer_config.json b/best/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..770e41d6c92519d525eede4cbcf3ba27f6425311
--- /dev/null
+++ b/best/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
diff --git a/checkpoint-100/README.md b/checkpoint-100/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..dd49221a54a197d9f0cab86c7982e2b2fe0776b3
--- /dev/null
+++ b/checkpoint-100/README.md
@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/checkpoint-100/adapter_config.json b/checkpoint-100/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..07d800af2efdcfa371d6bb94e3b2ed9781fce6cc
--- /dev/null
+++ b/checkpoint-100/adapter_config.json
@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-100/adapter_model.safetensors b/checkpoint-100/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1c50787494e7c836a23bf82c98bfa7788c4b4b03
--- /dev/null
+++ b/checkpoint-100/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47c56d30914398e534459320b96ed5f3e01c1d75acf5a864c12e908b4d4fcf98
+size 80792880
diff --git a/checkpoint-100/chat_template.jinja b/checkpoint-100/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/checkpoint-100/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/checkpoint-100/optimizer.pt b/checkpoint-100/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..a7584f86b71ee79a7515d68d2c001ede44be3b83
--- /dev/null
+++ b/checkpoint-100/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da9d3b2b7ccda4c29c7696ef388afee6ec7fe0a72f8ba7d069c29a78db44cddf
+size 161810282
diff --git a/checkpoint-100/rng_state.pth b/checkpoint-100/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..e3f1be8a6eab1b0784cbd035a7fdde6d95fdc9ce
--- /dev/null
+++ b/checkpoint-100/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:251ed89abdc9cbb2b330364ad2e1ed3c5952a8ca0e88b277da28300fb8eafd64
+size 14244
diff --git a/checkpoint-100/scheduler.pt b/checkpoint-100/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..920fb7563848c935700d7e0fc7d32018a17deec6
--- /dev/null
+++ b/checkpoint-100/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4046ef102a44bd51e356a969cc629e7433fb33305dc9006ab3b0360b40b90870
+size 1064
diff --git a/checkpoint-100/tokenizer.json b/checkpoint-100/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..34510ff0037cd50428af467a17ead5a96140a32c
--- /dev/null
+++ b/checkpoint-100/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
diff --git a/checkpoint-100/tokenizer_config.json b/checkpoint-100/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..770e41d6c92519d525eede4cbcf3ba27f6425311
--- /dev/null
+++ b/checkpoint-100/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
diff --git a/checkpoint-100/trainer_state.json b/checkpoint-100/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..5424040687b4ee2bee7815735e369c1cf6c8bfa4
--- /dev/null
+++ b/checkpoint-100/trainer_state.json
@@ -0,0 +1,95 @@
+{
+  "best_global_step": 100,
+  "best_metric": 0.3281489610671997,
+  "best_model_checkpoint": "/workspace/andy-D1-andyft/checkpoint-100",
+  "epoch": 0.1839503334099793,
+  "eval_steps": 100,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.358754662796855,
+      "epoch": 0.03679006668199586,
+      "grad_norm": 0.322265625,
+      "learning_rate": 6.785714285714286e-05,
+      "loss": 1.9387958526611329,
+      "mean_token_accuracy": 0.6328379280865193,
+      "num_tokens": 552605.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.2090021014213561,
+      "epoch": 0.07358013336399172,
+      "grad_norm": 0.3359375,
+      "learning_rate": 9.988791089826816e-05,
+      "loss": 1.0881237983703613,
+      "mean_token_accuracy": 0.7621329009532929,
+      "num_tokens": 1118812.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.5365577314049006,
+      "epoch": 0.11037020004598758,
+      "grad_norm": 0.265625,
+      "learning_rate": 9.91120793909987e-05,
+      "loss": 0.5044384002685547,
+      "mean_token_accuracy": 0.8858215417712927,
+      "num_tokens": 1693574.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.4165896608028561,
+      "epoch": 0.14716026672798344,
+      "grad_norm": 0.2451171875,
+      "learning_rate": 9.760895058620235e-05,
+      "loss": 0.38671829700469973,
+      "mean_token_accuracy": 0.9098233781754971,
+      "num_tokens": 2267338.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.3438017311505973,
+      "epoch": 0.1839503334099793,
+      "grad_norm": 0.2197265625,
+      "learning_rate": 9.540078421165121e-05,
+      "loss": 0.3093204736709595,
+      "mean_token_accuracy": 0.9240799810737371,
+      "num_tokens": 2850973.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.1839503334099793,
+      "eval_entropy": 0.33466541236795194,
+      "eval_loss": 0.3281489610671997,
+      "eval_mean_token_accuracy": 0.9268425211115175,
+      "eval_num_tokens": 2850973.0,
+      "eval_runtime": 156.4612,
+      "eval_samples_per_second": 2.921,
+      "eval_steps_per_second": 1.464,
+      "step": 100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 544,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.366754616492626e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-100/training_args.bin b/checkpoint-100/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..f1eb5ec9b5b40eee38c3620f7f396da225f9d3fe
--- /dev/null
+++ b/checkpoint-100/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9b517ebf9074949314ba9d100553b44cb8ea1809bee2c6d00f5dba2c2462537
+size 5240
diff --git a/checkpoint-200/README.md b/checkpoint-200/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..dd49221a54a197d9f0cab86c7982e2b2fe0776b3
--- /dev/null
+++ b/checkpoint-200/README.md
@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/checkpoint-200/adapter_config.json b/checkpoint-200/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..07d800af2efdcfa371d6bb94e3b2ed9781fce6cc
--- /dev/null
+++ b/checkpoint-200/adapter_config.json
@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-200/adapter_model.safetensors b/checkpoint-200/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..96ab5fbe8dad45a42106be4ed0bb80d3fb578cde
--- /dev/null
+++ b/checkpoint-200/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44ec8aab3cf9eb0c908fe2e86ceaa91a0ff47853e44232be81d0d3a336280ddb
+size 80792880
diff --git a/checkpoint-200/chat_template.jinja b/checkpoint-200/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/checkpoint-200/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/checkpoint-200/optimizer.pt b/checkpoint-200/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..61395665255e4beb95618ef809db73ecf43e7e24
--- /dev/null
+++ b/checkpoint-200/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a21d3f9ff06ddc4d695f8ca07fb0ea40e8746fa22948215ab9f9c8fda6f440e
+size 161810282
diff --git a/checkpoint-200/rng_state.pth b/checkpoint-200/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..38ea4fb39e4dbb37cfbe36ce144a19d78d880a00
--- /dev/null
+++ b/checkpoint-200/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:297c13f8d98c33e0a7d81994002fe9ea03bdc4d82549bad743c616ce10c95216
+size 14244
diff --git a/checkpoint-200/scheduler.pt b/checkpoint-200/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..431e23f2ffbc2c9ee3eeeb115ed2914a595483dd
--- /dev/null
+++ b/checkpoint-200/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1219abde915c9cc7a07aaa50b3f6254d6d84b5343dcbf23c1ac7b3076d8ebb1a
+size 1064
diff --git a/checkpoint-200/tokenizer.json b/checkpoint-200/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..34510ff0037cd50428af467a17ead5a96140a32c
--- /dev/null
+++ b/checkpoint-200/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
diff --git a/checkpoint-200/tokenizer_config.json b/checkpoint-200/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..770e41d6c92519d525eede4cbcf3ba27f6425311
--- /dev/null
+++ b/checkpoint-200/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
diff --git a/checkpoint-200/trainer_state.json b/checkpoint-200/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..c4e0559b5ff0eb79ab4566250b9989b875016ca2
--- /dev/null
+++ b/checkpoint-200/trainer_state.json
@@ -0,0 +1,156 @@
+{
+  "best_global_step": 200,
+  "best_metric": 0.24931415915489197,
+  "best_model_checkpoint": "/workspace/andy-D1-andyft/checkpoint-200",
+  "epoch": 0.3679006668199586,
+  "eval_steps": 100,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.358754662796855,
+      "epoch": 0.03679006668199586,
+      "grad_norm": 0.322265625,
+      "learning_rate": 6.785714285714286e-05,
+      "loss": 1.9387958526611329,
+      "mean_token_accuracy": 0.6328379280865193,
+      "num_tokens": 552605.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.2090021014213561,
+      "epoch": 0.07358013336399172,
+      "grad_norm": 0.3359375,
+      "learning_rate": 9.988791089826816e-05,
+      "loss": 1.0881237983703613,
+      "mean_token_accuracy": 0.7621329009532929,
+      "num_tokens": 1118812.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.5365577314049006,
+      "epoch": 0.11037020004598758,
+      "grad_norm": 0.265625,
+      "learning_rate": 9.91120793909987e-05,
+      "loss": 0.5044384002685547,
+      "mean_token_accuracy": 0.8858215417712927,
+      "num_tokens": 1693574.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.4165896608028561,
+      "epoch": 0.14716026672798344,
+      "grad_norm": 0.2451171875,
+      "learning_rate": 9.760895058620235e-05,
+      "loss": 0.38671829700469973,
+      "mean_token_accuracy": 0.9098233781754971,
+      "num_tokens": 2267338.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.3438017311505973,
+      "epoch": 0.1839503334099793,
+      "grad_norm": 0.2197265625,
+      "learning_rate": 9.540078421165121e-05,
+      "loss": 0.3093204736709595,
+      "mean_token_accuracy": 0.9240799810737371,
+      "num_tokens": 2850973.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.1839503334099793,
+      "eval_entropy": 0.33466541236795194,
+      "eval_loss": 0.3281489610671997,
+      "eval_mean_token_accuracy": 0.9268425211115175,
+      "eval_num_tokens": 2850973.0,
+      "eval_runtime": 156.4612,
+      "eval_samples_per_second": 2.921,
+      "eval_steps_per_second": 1.464,
+      "step": 100
+    },
+    {
+      "entropy": 0.3134843219071627,
+      "epoch": 0.22074040009197515,
+      "grad_norm": 0.189453125,
+      "learning_rate": 9.252028084644137e-05,
+      "loss": 0.28039026260375977,
+      "mean_token_accuracy": 0.92984459400177,
+      "num_tokens": 3429327.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.3144511237274855,
+      "epoch": 0.257530466773971,
+      "grad_norm": 0.22265625,
+      "learning_rate": 8.901009766043847e-05,
+      "loss": 0.27316670417785643,
+      "mean_token_accuracy": 0.9302996207028628,
+      "num_tokens": 3995502.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.28906950131058695,
+      "epoch": 0.29432053345596687,
+      "grad_norm": 0.267578125,
+      "learning_rate": 8.492221670727694e-05,
+      "loss": 0.2562049150466919,
+      "mean_token_accuracy": 0.9345210112631321,
+      "num_tokens": 4561185.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.309068296616897,
+      "epoch": 0.33111060013796273,
+      "grad_norm": 0.255859375,
+      "learning_rate": 8.031717512581784e-05,
+      "loss": 0.2678884267807007,
+      "mean_token_accuracy": 0.9299660053104162,
+      "num_tokens": 5133715.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.27301658713258803,
+      "epoch": 0.3679006668199586,
+      "grad_norm": 0.35546875,
+      "learning_rate": 7.526316864995647e-05,
+      "loss": 0.23800337314605713,
+      "mean_token_accuracy": 0.938116031512618,
+      "num_tokens": 5698771.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.3679006668199586,
+      "eval_entropy": 0.261124916667501,
+      "eval_loss": 0.24931415915489197,
+      "eval_mean_token_accuracy": 0.9429609785954505,
+      "eval_num_tokens": 5698771.0,
+      "eval_runtime": 156.272,
+      "eval_samples_per_second": 2.924,
+      "eval_steps_per_second": 1.465,
+      "step": 200
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 544,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.7278687962629734e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-200/training_args.bin b/checkpoint-200/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..f1eb5ec9b5b40eee38c3620f7f396da225f9d3fe
--- /dev/null
+++ b/checkpoint-200/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9b517ebf9074949314ba9d100553b44cb8ea1809bee2c6d00f5dba2c2462537
+size 5240
diff --git a/checkpoint-300/README.md b/checkpoint-300/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..dd49221a54a197d9f0cab86c7982e2b2fe0776b3
--- /dev/null
+++ b/checkpoint-300/README.md
@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/checkpoint-300/adapter_config.json b/checkpoint-300/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..07d800af2efdcfa371d6bb94e3b2ed9781fce6cc
--- /dev/null
+++ b/checkpoint-300/adapter_config.json
@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-300/adapter_model.safetensors b/checkpoint-300/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..6c3dceda90c4a1ea4d76ddf8a448d6d73ac38405
--- /dev/null
+++ b/checkpoint-300/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25b690434326cc9985a2be10aed0b86d9ff663bf9011a0f8e9ef0e9e66538b04
+size 80792880
diff --git a/checkpoint-300/chat_template.jinja b/checkpoint-300/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/checkpoint-300/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/checkpoint-300/optimizer.pt b/checkpoint-300/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..3233f837cf11f9a1046952b972657c165a80a8ce
--- /dev/null
+++ b/checkpoint-300/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a362d734701340d4467fb6aafe46d7b74500dbb4fb93a8ba8133ca469c698542
+size 161810282
diff --git a/checkpoint-300/rng_state.pth b/checkpoint-300/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..6bb542e635ef92e3dd8ccc17342eadcfb9deec49
--- /dev/null
+++ b/checkpoint-300/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55f8e34159ff42165225f7b3e516cdb75c945cbc647a6ed70e0cec051405061e
+size 14244
diff --git a/checkpoint-300/scheduler.pt b/checkpoint-300/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..323c16d9dfe9689b12b76d53d33b29c84de3ba0f
--- /dev/null
+++ b/checkpoint-300/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:899b22f1e1646600014811952841e18f5abf0e007125914547fcf11d3beb0b05
+size 1064
diff --git a/checkpoint-300/tokenizer.json b/checkpoint-300/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..34510ff0037cd50428af467a17ead5a96140a32c
--- /dev/null
+++ b/checkpoint-300/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
diff --git a/checkpoint-300/tokenizer_config.json b/checkpoint-300/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..770e41d6c92519d525eede4cbcf3ba27f6425311
--- /dev/null
+++ b/checkpoint-300/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
diff --git a/checkpoint-300/trainer_state.json b/checkpoint-300/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..8dbe681cd39183d0d18d3dd93b1fa17b4fb67011
--- /dev/null
+++ b/checkpoint-300/trainer_state.json
@@ -0,0 +1,217 @@
+{
+  "best_global_step": 300,
+  "best_metric": 0.2132847160100937,
+  "best_model_checkpoint": "/workspace/andy-D1-andyft/checkpoint-300",
+  "epoch": 0.5518510002299379,
+  "eval_steps": 100,
+  "global_step": 300,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.358754662796855,
+      "epoch": 0.03679006668199586,
+      "grad_norm": 0.322265625,
+      "learning_rate": 6.785714285714286e-05,
+      "loss": 1.9387958526611329,
+      "mean_token_accuracy": 0.6328379280865193,
+      "num_tokens": 552605.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.2090021014213561,
+      "epoch": 0.07358013336399172,
+      "grad_norm": 0.3359375,
+      "learning_rate": 9.988791089826816e-05,
+      "loss": 1.0881237983703613,
+      "mean_token_accuracy": 0.7621329009532929,
+      "num_tokens": 1118812.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.5365577314049006,
+      "epoch": 0.11037020004598758,
+      "grad_norm": 0.265625,
+      "learning_rate": 9.91120793909987e-05,
+      "loss": 0.5044384002685547,
+      "mean_token_accuracy": 0.8858215417712927,
+      "num_tokens": 1693574.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.4165896608028561,
+      "epoch": 0.14716026672798344,
+      "grad_norm": 0.2451171875,
+      "learning_rate": 9.760895058620235e-05,
+      "loss": 0.38671829700469973,
+      "mean_token_accuracy": 0.9098233781754971,
+      "num_tokens": 2267338.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.3438017311505973,
+      "epoch": 0.1839503334099793,
+      "grad_norm": 0.2197265625,
+      "learning_rate": 9.540078421165121e-05,
+      "loss": 0.3093204736709595,
+      "mean_token_accuracy": 0.9240799810737371,
+      "num_tokens": 2850973.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.1839503334099793,
+      "eval_entropy": 0.33466541236795194,
+      "eval_loss": 0.3281489610671997,
+      "eval_mean_token_accuracy": 0.9268425211115175,
+      "eval_num_tokens": 2850973.0,
+      "eval_runtime": 156.4612,
+      "eval_samples_per_second": 2.921,
+      "eval_steps_per_second": 1.464,
+      "step": 100
+    },
+    {
+      "entropy": 0.3134843219071627,
+      "epoch": 0.22074040009197515,
+      "grad_norm": 0.189453125,
+      "learning_rate": 9.252028084644137e-05,
+      "loss": 0.28039026260375977,
+      "mean_token_accuracy": 0.92984459400177,
+      "num_tokens": 3429327.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.3144511237274855,
+      "epoch": 0.257530466773971,
+      "grad_norm": 0.22265625,
+      "learning_rate": 8.901009766043847e-05,
+      "loss": 0.27316670417785643,
+      "mean_token_accuracy": 0.9302996207028628,
+      "num_tokens": 3995502.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.28906950131058695,
+      "epoch": 0.29432053345596687,
+      "grad_norm": 0.267578125,
+      "learning_rate": 8.492221670727694e-05,
+      "loss": 0.2562049150466919,
+      "mean_token_accuracy": 0.9345210112631321,
+      "num_tokens": 4561185.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.309068296616897,
+      "epoch": 0.33111060013796273,
+      "grad_norm": 0.255859375,
+      "learning_rate": 8.031717512581784e-05,
+      "loss": 0.2678884267807007,
+      "mean_token_accuracy": 0.9299660053104162,
+      "num_tokens": 5133715.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.27301658713258803,
+      "epoch": 0.3679006668199586,
+      "grad_norm": 0.35546875,
+      "learning_rate": 7.526316864995647e-05,
+      "loss": 0.23800337314605713,
+      "mean_token_accuracy": 0.938116031512618,
+      "num_tokens": 5698771.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.3679006668199586,
+      "eval_entropy": 0.261124916667501,
+      "eval_loss": 0.24931415915489197,
+      "eval_mean_token_accuracy": 0.9429609785954505,
+      "eval_num_tokens": 5698771.0,
+      "eval_runtime": 156.272,
+      "eval_samples_per_second": 2.924,
+      "eval_steps_per_second": 1.465,
+      "step": 200
+    },
+    {
+      "entropy": 0.2801984048448503,
+      "epoch": 0.40469073350195445,
+      "grad_norm": 0.294921875,
+      "learning_rate": 6.983504170283999e-05,
+      "loss": 0.2422886610031128,
+      "mean_token_accuracy": 0.9372635453939437,
+      "num_tokens": 6263699.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.2785345103126019,
+      "epoch": 0.4414808001839503,
+      "grad_norm": 0.2578125,
+      "learning_rate": 6.411317903111779e-05,
+      "loss": 0.24179139137268066,
+      "mean_token_accuracy": 0.9373910058289766,
+      "num_tokens": 6825294.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.2691320342477411,
+      "epoch": 0.4782708668659462,
+      "grad_norm": 0.2578125,
+      "learning_rate": 5.8182315292934406e-05,
+      "loss": 0.23189406394958495,
+      "mean_token_accuracy": 0.9388320956379175,
+      "num_tokens": 7391294.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.24280655251350253,
+      "epoch": 0.515060933547942,
+      "grad_norm": 0.29296875,
+      "learning_rate": 5.213028022839188e-05,
+      "loss": 0.20993587970733643,
+      "mean_token_accuracy": 0.9451322961598635,
+      "num_tokens": 7949107.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.24249689073767514,
+      "epoch": 0.5518510002299379,
+      "grad_norm": 0.275390625,
+      "learning_rate": 4.6046697995163003e-05,
+      "loss": 0.21177098751068116,
+      "mean_token_accuracy": 0.9459892265498638,
+      "num_tokens": 8526495.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.5518510002299379,
+      "eval_entropy": 0.22384919074880505,
+      "eval_loss": 0.2132847160100937,
+      "eval_mean_token_accuracy": 0.9506369665200013,
+      "eval_num_tokens": 8526495.0,
+      "eval_runtime": 156.3249,
+      "eval_samples_per_second": 2.923,
+      "eval_steps_per_second": 1.465,
+      "step": 300
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 544,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.086527978083246e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-300/training_args.bin b/checkpoint-300/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..f1eb5ec9b5b40eee38c3620f7f396da225f9d3fe
--- /dev/null
+++ b/checkpoint-300/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9b517ebf9074949314ba9d100553b44cb8ea1809bee2c6d00f5dba2c2462537
+size 5240
diff --git a/checkpoint-400/README.md b/checkpoint-400/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..dd49221a54a197d9f0cab86c7982e2b2fe0776b3
--- /dev/null
+++ b/checkpoint-400/README.md
@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/checkpoint-400/adapter_config.json b/checkpoint-400/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..07d800af2efdcfa371d6bb94e3b2ed9781fce6cc
--- /dev/null
+++ b/checkpoint-400/adapter_config.json
@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-400/adapter_model.safetensors b/checkpoint-400/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..815f06dcc5a34bd08459aa0fa9be22af656c9ca7
--- /dev/null
+++ b/checkpoint-400/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42a5b221c9ce1038d4d59cd9e000a8e6046d0614048406a528530fea0d82d78f
+size 80792880
diff --git a/checkpoint-400/chat_template.jinja b/checkpoint-400/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/checkpoint-400/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/checkpoint-400/optimizer.pt b/checkpoint-400/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..b39dea51345d0f4b958c13a5a71012f21213df1e
--- /dev/null
+++ b/checkpoint-400/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b019795e893572410d4be8f1e4e3c6387407c256ebeb22c5c9eacb4fdeb67c2e
+size 161810282
diff --git a/checkpoint-400/rng_state.pth b/checkpoint-400/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..ed9cd34f98199471916bd69caaa7e2083af2a289
--- /dev/null
+++ b/checkpoint-400/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f782068884210c9289b175d2c0bfdc20bd2c6de8bffa2ae98fc2dcf0086ba48f
+size 14244
diff --git a/checkpoint-400/scheduler.pt b/checkpoint-400/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..97003f302d5417decb6616078508d1bb7a5ce7bc
--- /dev/null
+++ b/checkpoint-400/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:038eb4e53262d9fb232f392040e8cb077b734b4bf964fa822e739d5271a82c4e
+size 1064
diff --git a/checkpoint-400/tokenizer.json b/checkpoint-400/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..34510ff0037cd50428af467a17ead5a96140a32c
--- /dev/null
+++ b/checkpoint-400/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
diff --git a/checkpoint-400/tokenizer_config.json b/checkpoint-400/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..770e41d6c92519d525eede4cbcf3ba27f6425311
--- /dev/null
+++ b/checkpoint-400/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
diff --git a/checkpoint-400/trainer_state.json b/checkpoint-400/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..6b5845e9ee97438385cd79c6dabf5802286fd434
--- /dev/null
+++ b/checkpoint-400/trainer_state.json
@@ -0,0 +1,278 @@
+{
+  "best_global_step": 400,
+  "best_metric": 0.19629092514514923,
+  "best_model_checkpoint": "/workspace/andy-D1-andyft/checkpoint-400",
+  "epoch": 0.7358013336399172,
+  "eval_steps": 100,
+  "global_step": 400,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.358754662796855,
+      "epoch": 0.03679006668199586,
+      "grad_norm": 0.322265625,
+      "learning_rate": 6.785714285714286e-05,
+      "loss": 1.9387958526611329,
+      "mean_token_accuracy": 0.6328379280865193,
+      "num_tokens": 552605.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.2090021014213561,
+      "epoch": 0.07358013336399172,
+      "grad_norm": 0.3359375,
+      "learning_rate": 9.988791089826816e-05,
+      "loss": 1.0881237983703613,
+      "mean_token_accuracy": 0.7621329009532929,
+      "num_tokens": 1118812.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.5365577314049006,
+      "epoch": 0.11037020004598758,
+      "grad_norm": 0.265625,
+      "learning_rate": 9.91120793909987e-05,
+      "loss": 0.5044384002685547,
+      "mean_token_accuracy": 0.8858215417712927,
+      "num_tokens": 1693574.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.4165896608028561,
+      "epoch": 0.14716026672798344,
+      "grad_norm": 0.2451171875,
+      "learning_rate": 9.760895058620235e-05,
+      "loss": 0.38671829700469973,
+      "mean_token_accuracy": 0.9098233781754971,
+      "num_tokens": 2267338.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.3438017311505973,
+      "epoch": 0.1839503334099793,
+      "grad_norm": 0.2197265625,
+      "learning_rate": 9.540078421165121e-05,
+      "loss": 0.3093204736709595,
+      "mean_token_accuracy": 0.9240799810737371,
+      "num_tokens": 2850973.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.1839503334099793,
+      "eval_entropy": 0.33466541236795194,
+      "eval_loss": 0.3281489610671997,
+      "eval_mean_token_accuracy": 0.9268425211115175,
+      "eval_num_tokens": 2850973.0,
+      "eval_runtime": 156.4612,
+      "eval_samples_per_second": 2.921,
+      "eval_steps_per_second": 1.464,
+      "step": 100
+    },
+    {
+      "entropy": 0.3134843219071627,
+      "epoch": 0.22074040009197515,
+      "grad_norm": 0.189453125,
+      "learning_rate": 9.252028084644137e-05,
+      "loss": 0.28039026260375977,
+      "mean_token_accuracy": 0.92984459400177,
+      "num_tokens": 3429327.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.3144511237274855,
+      "epoch": 0.257530466773971,
+      "grad_norm": 0.22265625,
+      "learning_rate": 8.901009766043847e-05,
+      "loss": 0.27316670417785643,
+      "mean_token_accuracy": 0.9302996207028628,
+      "num_tokens": 3995502.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.28906950131058695,
+      "epoch": 0.29432053345596687,
+      "grad_norm": 0.267578125,
+      "learning_rate": 8.492221670727694e-05,
+      "loss": 0.2562049150466919,
+      "mean_token_accuracy": 0.9345210112631321,
+      "num_tokens": 4561185.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.309068296616897,
+      "epoch": 0.33111060013796273,
+      "grad_norm": 0.255859375,
+      "learning_rate": 8.031717512581784e-05,
+      "loss": 0.2678884267807007,
+      "mean_token_accuracy": 0.9299660053104162,
+      "num_tokens": 5133715.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.27301658713258803,
+      "epoch": 0.3679006668199586,
+      "grad_norm": 0.35546875,
+      "learning_rate": 7.526316864995647e-05,
+      "loss": 0.23800337314605713,
+      "mean_token_accuracy": 0.938116031512618,
+      "num_tokens": 5698771.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.3679006668199586,
+      "eval_entropy": 0.261124916667501,
+      "eval_loss": 0.24931415915489197,
+      "eval_mean_token_accuracy": 0.9429609785954505,
+      "eval_num_tokens": 5698771.0,
+      "eval_runtime": 156.272,
+      "eval_samples_per_second": 2.924,
+      "eval_steps_per_second": 1.465,
+      "step": 200
+    },
+    {
+      "entropy": 0.2801984048448503,
+      "epoch": 0.40469073350195445,
+      "grad_norm": 0.294921875,
+      "learning_rate": 6.983504170283999e-05,
+      "loss": 0.2422886610031128,
+      "mean_token_accuracy": 0.9372635453939437,
+      "num_tokens": 6263699.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.2785345103126019,
+      "epoch": 0.4414808001839503,
+      "grad_norm": 0.2578125,
+      "learning_rate": 6.411317903111779e-05,
+      "loss": 0.24179139137268066,
+      "mean_token_accuracy": 0.9373910058289766,
+      "num_tokens": 6825294.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.2691320342477411,
+      "epoch": 0.4782708668659462,
+      "grad_norm": 0.2578125,
+      "learning_rate": 5.8182315292934406e-05,
+      "loss": 0.23189406394958495,
+      "mean_token_accuracy": 0.9388320956379175,
+      "num_tokens": 7391294.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.24280655251350253,
+      "epoch": 0.515060933547942,
+      "grad_norm": 0.29296875,
+      "learning_rate": 5.213028022839188e-05,
+      "loss": 0.20993587970733643,
+      "mean_token_accuracy": 0.9451322961598635,
+      "num_tokens": 7949107.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.24249689073767514,
+      "epoch": 0.5518510002299379,
+      "grad_norm": 0.275390625,
+      "learning_rate": 4.6046697995163003e-05,
+      "loss": 0.21177098751068116,
+      "mean_token_accuracy": 0.9459892265498638,
+      "num_tokens": 8526495.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.5518510002299379,
+      "eval_entropy": 0.22384919074880505,
+      "eval_loss": 0.2132847160100937,
+      "eval_mean_token_accuracy": 0.9506369665200013,
+      "eval_num_tokens": 8526495.0,
+      "eval_runtime": 156.3249,
+      "eval_samples_per_second": 2.923,
+      "eval_steps_per_second": 1.465,
+      "step": 300
+    },
+    {
+      "entropy": 0.21746529480442406,
+      "epoch": 0.5886410669119337,
+      "grad_norm": 0.27734375,
+      "learning_rate": 4.002165993070237e-05,
+      "loss": 0.1864999294281006,
+      "mean_token_accuracy": 0.9523148186504841,
+      "num_tokens": 9103591.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.22349709873087703,
+      "epoch": 0.6254311335939297,
+      "grad_norm": 0.26953125,
+      "learning_rate": 3.414439039602495e-05,
+      "loss": 0.18752355575561525,
+      "mean_token_accuracy": 0.9500059768557548,
+      "num_tokens": 9677492.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.20582567660603673,
+      "epoch": 0.6622212002759255,
+      "grad_norm": 0.3203125,
+      "learning_rate": 2.8501925458477353e-05,
+      "loss": 0.1842264413833618,
+      "mean_token_accuracy": 0.9537915319204331,
+      "num_tokens": 10255825.0,
+      "step": 360
+    },
+    {
+      "entropy": 0.2267866414040327,
+      "epoch": 0.6990112669579214,
+      "grad_norm": 0.27734375,
+      "learning_rate": 2.3177823980794727e-05,
+      "loss": 0.19315229654312133,
+      "mean_token_accuracy": 0.9484861396253109,
+      "num_tokens": 10829988.0,
+      "step": 380
+    },
+    {
+      "entropy": 0.23870108206756413,
+      "epoch": 0.7358013336399172,
+      "grad_norm": 0.279296875,
+      "learning_rate": 1.8250930203833732e-05,
+      "loss": 0.2072918176651001,
+      "mean_token_accuracy": 0.946313539147377,
+      "num_tokens": 11399434.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.7358013336399172,
+      "eval_entropy": 0.20661315126710583,
+      "eval_loss": 0.19629092514514923,
+      "eval_mean_token_accuracy": 0.9545749706992936,
+      "eval_num_tokens": 11399434.0,
+      "eval_runtime": 155.9996,
+      "eval_samples_per_second": 2.929,
+      "eval_steps_per_second": 1.468,
+      "step": 400
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 544,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.45469995696554e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-400/training_args.bin b/checkpoint-400/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..f1eb5ec9b5b40eee38c3620f7f396da225f9d3fe
--- /dev/null
+++ b/checkpoint-400/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9b517ebf9074949314ba9d100553b44cb8ea1809bee2c6d00f5dba2c2462537
+size 5240
diff --git a/checkpoint-500/README.md b/checkpoint-500/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..dd49221a54a197d9f0cab86c7982e2b2fe0776b3
--- /dev/null
+++ b/checkpoint-500/README.md
@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/checkpoint-500/adapter_config.json b/checkpoint-500/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..07d800af2efdcfa371d6bb94e3b2ed9781fce6cc
--- /dev/null
+++ b/checkpoint-500/adapter_config.json
@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-500/adapter_model.safetensors b/checkpoint-500/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..593e2ce6e33fe7e9a7aa91b99f8f48120ceb0175
--- /dev/null
+++ b/checkpoint-500/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c463ef2e5863b1bc368396ceff41c92ecb15d92ceddaf419bb97d492efe198c
+size 80792880
diff --git a/checkpoint-500/chat_template.jinja b/checkpoint-500/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/checkpoint-500/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/checkpoint-500/optimizer.pt b/checkpoint-500/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..b54e165787fb4ac83d5f59bcc9e4c5aa5d8e0ada
--- /dev/null
+++ b/checkpoint-500/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14a5155c40c88abbc6553636f8e3acfc55fb640a0837fbc9283dca6414e04695
+size 161810282
diff --git a/checkpoint-500/rng_state.pth b/checkpoint-500/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..bef332ff645cbccdf82e18e1011e7c56726d26f1
--- /dev/null
+++ b/checkpoint-500/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f698943f76819f538f031210aeb6388fb2393901a6d9fd9c04455657b0e2b1d
+size 14244
diff --git a/checkpoint-500/scheduler.pt b/checkpoint-500/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..753f7ba67785ce14650895caa8ef687b3086b722
--- /dev/null
+++ b/checkpoint-500/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1d616c55b04214181f1959ac44d3142972e5420b3215597f81b9911a7889b78
+size 1064
diff --git a/checkpoint-500/tokenizer.json b/checkpoint-500/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..34510ff0037cd50428af467a17ead5a96140a32c
--- /dev/null
+++ b/checkpoint-500/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
diff --git a/checkpoint-500/tokenizer_config.json b/checkpoint-500/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..770e41d6c92519d525eede4cbcf3ba27f6425311
--- /dev/null
+++ b/checkpoint-500/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
diff --git a/checkpoint-500/trainer_state.json b/checkpoint-500/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..4ad91d016a5d000d0f1715568059cf4a59e497b5
--- /dev/null
+++ b/checkpoint-500/trainer_state.json
@@ -0,0 +1,339 @@
+{
+  "best_global_step": 500,
+  "best_metric": 0.19292491674423218,
+  "best_model_checkpoint": "/workspace/andy-D1-andyft/checkpoint-500",
+  "epoch": 0.9197516670498965,
+  "eval_steps": 100,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.358754662796855,
+      "epoch": 0.03679006668199586,
+      "grad_norm": 0.322265625,
+      "learning_rate": 6.785714285714286e-05,
+      "loss": 1.9387958526611329,
+      "mean_token_accuracy": 0.6328379280865193,
+      "num_tokens": 552605.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.2090021014213561,
+      "epoch": 0.07358013336399172,
+      "grad_norm": 0.3359375,
+      "learning_rate": 9.988791089826816e-05,
+      "loss": 1.0881237983703613,
+      "mean_token_accuracy": 0.7621329009532929,
+      "num_tokens": 1118812.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.5365577314049006,
+      "epoch": 0.11037020004598758,
+      "grad_norm": 0.265625,
+      "learning_rate": 9.91120793909987e-05,
+      "loss": 0.5044384002685547,
+      "mean_token_accuracy": 0.8858215417712927,
+      "num_tokens": 1693574.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.4165896608028561,
+      "epoch": 0.14716026672798344,
+      "grad_norm": 0.2451171875,
+      "learning_rate": 9.760895058620235e-05,
+      "loss": 0.38671829700469973,
+      "mean_token_accuracy": 0.9098233781754971,
+      "num_tokens": 2267338.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.3438017311505973,
+      "epoch": 0.1839503334099793,
+      "grad_norm": 0.2197265625,
+      "learning_rate": 9.540078421165121e-05,
+      "loss": 0.3093204736709595,
+      "mean_token_accuracy": 0.9240799810737371,
+      "num_tokens": 2850973.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.1839503334099793,
+      "eval_entropy": 0.33466541236795194,
+      "eval_loss": 0.3281489610671997,
+      "eval_mean_token_accuracy": 0.9268425211115175,
+      "eval_num_tokens": 2850973.0,
+      "eval_runtime": 156.4612,
+      "eval_samples_per_second": 2.921,
+      "eval_steps_per_second": 1.464,
+      "step": 100
+    },
+    {
+      "entropy": 0.3134843219071627,
+      "epoch": 0.22074040009197515,
+      "grad_norm": 0.189453125,
+      "learning_rate": 9.252028084644137e-05,
+      "loss": 0.28039026260375977,
+      "mean_token_accuracy": 0.92984459400177,
+      "num_tokens": 3429327.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.3144511237274855,
+      "epoch": 0.257530466773971,
+      "grad_norm": 0.22265625,
+      "learning_rate": 8.901009766043847e-05,
+      "loss": 0.27316670417785643,
+      "mean_token_accuracy": 0.9302996207028628,
+      "num_tokens": 3995502.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.28906950131058695,
+      "epoch": 0.29432053345596687,
+      "grad_norm": 0.267578125,
+      "learning_rate": 8.492221670727694e-05,
+      "loss": 0.2562049150466919,
+      "mean_token_accuracy": 0.9345210112631321,
+      "num_tokens": 4561185.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.309068296616897,
+      "epoch": 0.33111060013796273,
+      "grad_norm": 0.255859375,
+      "learning_rate": 8.031717512581784e-05,
+      "loss": 0.2678884267807007,
+      "mean_token_accuracy": 0.9299660053104162,
+      "num_tokens": 5133715.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.27301658713258803,
+      "epoch": 0.3679006668199586,
+      "grad_norm": 0.35546875,
+      "learning_rate": 7.526316864995647e-05,
+      "loss": 0.23800337314605713,
+      "mean_token_accuracy": 0.938116031512618,
+      "num_tokens": 5698771.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.3679006668199586,
+      "eval_entropy": 0.261124916667501,
+      "eval_loss": 0.24931415915489197,
+      "eval_mean_token_accuracy": 0.9429609785954505,
+      "eval_num_tokens": 5698771.0,
+      "eval_runtime": 156.272,
+      "eval_samples_per_second": 2.924,
+      "eval_steps_per_second": 1.465,
+      "step": 200
+    },
+    {
+      "entropy": 0.2801984048448503,
+      "epoch": 0.40469073350195445,
+      "grad_norm": 0.294921875,
+      "learning_rate": 6.983504170283999e-05,
+      "loss": 0.2422886610031128,
+      "mean_token_accuracy": 0.9372635453939437,
+      "num_tokens": 6263699.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.2785345103126019,
+      "epoch": 0.4414808001839503,
+      "grad_norm": 0.2578125,
+      "learning_rate": 6.411317903111779e-05,
+      "loss": 0.24179139137268066,
+      "mean_token_accuracy": 0.9373910058289766,
+      "num_tokens": 6825294.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.2691320342477411,
+      "epoch": 0.4782708668659462,
+      "grad_norm": 0.2578125,
+      "learning_rate": 5.8182315292934406e-05,
+      "loss": 0.23189406394958495,
+      "mean_token_accuracy": 0.9388320956379175,
+      "num_tokens": 7391294.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.24280655251350253,
+      "epoch": 0.515060933547942,
+      "grad_norm": 0.29296875,
+      "learning_rate": 5.213028022839188e-05,
+      "loss": 0.20993587970733643,
+      "mean_token_accuracy": 0.9451322961598635,
+      "num_tokens": 7949107.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.24249689073767514,
+      "epoch": 0.5518510002299379,
+      "grad_norm": 0.275390625,
+      "learning_rate": 4.6046697995163003e-05,
+      "loss": 0.21177098751068116,
+      "mean_token_accuracy": 0.9459892265498638,
+      "num_tokens": 8526495.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.5518510002299379,
+      "eval_entropy": 0.22384919074880505,
+      "eval_loss": 0.2132847160100937,
+      "eval_mean_token_accuracy": 0.9506369665200013,
+      "eval_num_tokens": 8526495.0,
+      "eval_runtime": 156.3249,
+      "eval_samples_per_second": 2.923,
+      "eval_steps_per_second": 1.465,
+      "step": 300
+    },
+    {
+      "entropy": 0.21746529480442406,
+      "epoch": 0.5886410669119337,
+      "grad_norm": 0.27734375,
+      "learning_rate": 4.002165993070237e-05,
+      "loss": 0.1864999294281006,
+      "mean_token_accuracy": 0.9523148186504841,
+      "num_tokens": 9103591.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.22349709873087703,
+      "epoch": 0.6254311335939297,
+      "grad_norm": 0.26953125,
+      "learning_rate": 3.414439039602495e-05,
+      "loss": 0.18752355575561525,
+      "mean_token_accuracy": 0.9500059768557548,
+      "num_tokens": 9677492.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.20582567660603673,
+      "epoch": 0.6622212002759255,
+      "grad_norm": 0.3203125,
+      "learning_rate": 2.8501925458477353e-05,
+      "loss": 0.1842264413833618,
+      "mean_token_accuracy": 0.9537915319204331,
+      "num_tokens": 10255825.0,
+      "step": 360
+    },
+    {
+      "entropy": 0.2267866414040327,
+      "epoch": 0.6990112669579214,
+      "grad_norm": 0.27734375,
+      "learning_rate": 2.3177823980794727e-05,
+      "loss": 0.19315229654312133,
+      "mean_token_accuracy": 0.9484861396253109,
+      "num_tokens": 10829988.0,
+      "step": 380
+    },
+    {
+      "entropy": 0.23870108206756413,
+      "epoch": 0.7358013336399172,
+      "grad_norm": 0.279296875,
+      "learning_rate": 1.8250930203833732e-05,
+      "loss": 0.2072918176651001,
+      "mean_token_accuracy": 0.946313539147377,
+      "num_tokens": 11399434.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.7358013336399172,
+      "eval_entropy": 0.20661315126710583,
+      "eval_loss": 0.19629092514514923,
+      "eval_mean_token_accuracy": 0.9545749706992936,
+      "eval_num_tokens": 11399434.0,
+      "eval_runtime": 155.9996,
+      "eval_samples_per_second": 2.929,
+      "eval_steps_per_second": 1.468,
+      "step": 400
+    },
+    {
+      "entropy": 0.234044328937307,
+      "epoch": 0.7725914003219131,
+      "grad_norm": 0.3046875,
+      "learning_rate": 1.3794206147806521e-05,
+      "loss": 0.1922011137008667,
+      "mean_token_accuracy": 0.9485243733972311,
+      "num_tokens": 11970354.0,
+      "step": 420
+    },
+    {
+      "entropy": 0.2075159418163821,
+      "epoch": 0.8093814670039089,
+      "grad_norm": 0.275390625,
+      "learning_rate": 9.873651122902472e-06,
+      "loss": 0.18551079034805298,
+      "mean_token_accuracy": 0.9534976109862328,
+      "num_tokens": 12555199.0,
+      "step": 440
+    },
+    {
+      "entropy": 0.21931461119093001,
+      "epoch": 0.8461715336859048,
+      "grad_norm": 0.29296875,
+      "learning_rate": 6.547324350188477e-06,
+      "loss": 0.18772661685943604,
+      "mean_token_accuracy": 0.9516077276319266,
+      "num_tokens": 13124259.0,
+      "step": 460
+    },
+    {
+      "entropy": 0.20921103837899863,
+      "epoch": 0.8829616003679006,
+      "grad_norm": 0.26953125,
+      "learning_rate": 3.864485166725517e-06,
+      "loss": 0.18070039749145508,
+      "mean_token_accuracy": 0.9534818135201931,
+      "num_tokens": 13700247.0,
+      "step": 480
+    },
+    {
+      "entropy": 0.20970528924372048,
+      "epoch": 0.9197516670498965,
+      "grad_norm": 0.2353515625,
+      "learning_rate": 1.864863547542711e-06,
+      "loss": 0.17404462099075318,
+      "mean_token_accuracy": 0.9535064313560724,
+      "num_tokens": 14274173.0,
+      "step": 500
+    },
+    {
+      "epoch": 0.9197516670498965,
+      "eval_entropy": 0.20478792306377377,
+      "eval_loss": 0.19292491674423218,
+      "eval_mean_token_accuracy": 0.9554153168045277,
+      "eval_num_tokens": 14274173.0,
+      "eval_runtime": 156.1512,
+      "eval_samples_per_second": 2.927,
+      "eval_steps_per_second": 1.467,
+      "step": 500
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 544,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.823691121816576e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-500/training_args.bin b/checkpoint-500/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..f1eb5ec9b5b40eee38c3620f7f396da225f9d3fe
--- /dev/null
+++ b/checkpoint-500/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9b517ebf9074949314ba9d100553b44cb8ea1809bee2c6d00f5dba2c2462537
+size 5240
diff --git a/checkpoint-544/README.md b/checkpoint-544/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..dd49221a54a197d9f0cab86c7982e2b2fe0776b3
--- /dev/null
+++ b/checkpoint-544/README.md
@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1
\ No newline at end of file
diff --git a/checkpoint-544/adapter_config.json b/checkpoint-544/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..07d800af2efdcfa371d6bb94e3b2ed9781fce6cc
--- /dev/null
+++ b/checkpoint-544/adapter_config.json
@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-544/adapter_model.safetensors b/checkpoint-544/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..487a7ccaec0de2dce42e6a19abd2b72735e3b135
--- /dev/null
+++ b/checkpoint-544/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d327908bba3c5ae9ce40382642eff21179400a9822fabd1e86c42adde4a8f92a
+size 80792880
diff --git a/checkpoint-544/chat_template.jinja b/checkpoint-544/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7919a96cfe43d50914a007b9c0877bd0ec27e
--- /dev/null
+++ b/checkpoint-544/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/checkpoint-544/optimizer.pt b/checkpoint-544/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..7abf62c2b7e4cbe19a7a9fef675001a79d44547c
--- /dev/null
+++ b/checkpoint-544/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:756ee2adfbba4841e308b84247e21c025e8c1fb1171a02d5851ec74ff0132b1c
+size 161810282
diff --git a/checkpoint-544/rng_state.pth b/checkpoint-544/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..54ffb0ed62930f1dccb1adb0ae3b8965dcda73b4
--- /dev/null
+++ b/checkpoint-544/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10fb5d88c229dc9200c0d17b5da4d9e6ada6eef8bf997e63c3cf41d2cedb6022
+size 14244
diff --git a/checkpoint-544/scheduler.pt b/checkpoint-544/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..3c2f3e91b17dbabcf79f2f60d2bb712376355bfd
--- /dev/null
+++ b/checkpoint-544/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:745f4d0103b2900406e72d41dc56e739558e8d9bfc03dc26187f2046f54eae00
+size 1064
diff --git a/checkpoint-544/tokenizer.json b/checkpoint-544/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..34510ff0037cd50428af467a17ead5a96140a32c
--- /dev/null
+++ b/checkpoint-544/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
diff --git a/checkpoint-544/tokenizer_config.json b/checkpoint-544/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..770e41d6c92519d525eede4cbcf3ba27f6425311
--- /dev/null
+++ b/checkpoint-544/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
diff --git a/checkpoint-544/trainer_state.json b/checkpoint-544/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..5bb7e9b05aa8d2e941932fda15de390654965e18
--- /dev/null
+++ b/checkpoint-544/trainer_state.json
@@ -0,0 +1,370 @@
+{
+  "best_global_step": 544,
+  "best_metric": 0.19291424751281738,
+  "best_model_checkpoint": "/workspace/andy-D1-andyft/checkpoint-544",
+  "epoch": 1.0,
+  "eval_steps": 100,
+  "global_step": 544,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.358754662796855,
+      "epoch": 0.03679006668199586,
+      "grad_norm": 0.322265625,
+      "learning_rate": 6.785714285714286e-05,
+      "loss": 1.9387958526611329,
+      "mean_token_accuracy": 0.6328379280865193,
+      "num_tokens": 552605.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.2090021014213561,
+      "epoch": 0.07358013336399172,
+      "grad_norm": 0.3359375,
+      "learning_rate": 9.988791089826816e-05,
+      "loss": 1.0881237983703613,
+      "mean_token_accuracy": 0.7621329009532929,
+      "num_tokens": 1118812.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.5365577314049006,
+      "epoch": 0.11037020004598758,
+      "grad_norm": 0.265625,
+      "learning_rate": 9.91120793909987e-05,
+      "loss": 0.5044384002685547,
+      "mean_token_accuracy": 0.8858215417712927,
+      "num_tokens": 1693574.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.4165896608028561,
+      "epoch": 0.14716026672798344,
+      "grad_norm": 0.2451171875,
+      "learning_rate": 9.760895058620235e-05,
+      "loss": 0.38671829700469973,
+      "mean_token_accuracy": 0.9098233781754971,
+      "num_tokens": 2267338.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.3438017311505973,
+      "epoch": 0.1839503334099793,
+      "grad_norm": 0.2197265625,
+      "learning_rate": 9.540078421165121e-05,
+      "loss": 0.3093204736709595,
+      "mean_token_accuracy": 0.9240799810737371,
+      "num_tokens": 2850973.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.1839503334099793,
+      "eval_entropy": 0.33466541236795194,
+      "eval_loss": 0.3281489610671997,
+      "eval_mean_token_accuracy": 0.9268425211115175,
+      "eval_num_tokens": 2850973.0,
+      "eval_runtime": 156.4612,
+      "eval_samples_per_second": 2.921,
+      "eval_steps_per_second": 1.464,
+      "step": 100
+    },
+    {
+      "entropy": 0.3134843219071627,
+      "epoch": 0.22074040009197515,
+      "grad_norm": 0.189453125,
+      "learning_rate": 9.252028084644137e-05,
+      "loss": 0.28039026260375977,
+      "mean_token_accuracy": 0.92984459400177,
+      "num_tokens": 3429327.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.3144511237274855,
+      "epoch": 0.257530466773971,
+      "grad_norm": 0.22265625,
+      "learning_rate": 8.901009766043847e-05,
+      "loss": 0.27316670417785643,
+      "mean_token_accuracy": 0.9302996207028628,
+      "num_tokens": 3995502.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.28906950131058695,
+      "epoch": 0.29432053345596687,
+      "grad_norm": 0.267578125,
+      "learning_rate": 8.492221670727694e-05,
+      "loss": 0.2562049150466919,
+      "mean_token_accuracy": 0.9345210112631321,
+      "num_tokens": 4561185.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.309068296616897,
+      "epoch": 0.33111060013796273,
+      "grad_norm": 0.255859375,
+      "learning_rate": 8.031717512581784e-05,
+      "loss": 0.2678884267807007,
+      "mean_token_accuracy": 0.9299660053104162,
+      "num_tokens": 5133715.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.27301658713258803,
+      "epoch": 0.3679006668199586,
+      "grad_norm": 0.35546875,
+      "learning_rate": 7.526316864995647e-05,
+      "loss": 0.23800337314605713,
+      "mean_token_accuracy": 0.938116031512618,
+      "num_tokens": 5698771.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.3679006668199586,
+      "eval_entropy": 0.261124916667501,
+      "eval_loss": 0.24931415915489197,
+      "eval_mean_token_accuracy": 0.9429609785954505,
+      "eval_num_tokens": 5698771.0,
+      "eval_runtime": 156.272,
+      "eval_samples_per_second": 2.924,
+      "eval_steps_per_second": 1.465,
+      "step": 200
+    },
+    {
+      "entropy": 0.2801984048448503,
+      "epoch": 0.40469073350195445,
+      "grad_norm": 0.294921875,
+      "learning_rate": 6.983504170283999e-05,
+      "loss": 0.2422886610031128,
+      "mean_token_accuracy": 0.9372635453939437,
+      "num_tokens": 6263699.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.2785345103126019,
+      "epoch": 0.4414808001839503,
+      "grad_norm": 0.2578125,
+      "learning_rate": 6.411317903111779e-05,
+      "loss": 0.24179139137268066,
+      "mean_token_accuracy": 0.9373910058289766,
+      "num_tokens": 6825294.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.2691320342477411,
+      "epoch": 0.4782708668659462,
+      "grad_norm": 0.2578125,
+      "learning_rate": 5.8182315292934406e-05,
+      "loss": 0.23189406394958495,
+      "mean_token_accuracy": 0.9388320956379175,
+      "num_tokens": 7391294.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.24280655251350253,
+      "epoch": 0.515060933547942,
+      "grad_norm": 0.29296875,
+      "learning_rate": 5.213028022839188e-05,
+      "loss": 0.20993587970733643,
+      "mean_token_accuracy": 0.9451322961598635,
+      "num_tokens": 7949107.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.24249689073767514,
+      "epoch": 0.5518510002299379,
+      "grad_norm": 0.275390625,
+      "learning_rate": 4.6046697995163003e-05,
+      "loss": 0.21177098751068116,
+      "mean_token_accuracy": 0.9459892265498638,
+      "num_tokens": 8526495.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.5518510002299379,
+      "eval_entropy": 0.22384919074880505,
+      "eval_loss": 0.2132847160100937,
+      "eval_mean_token_accuracy": 0.9506369665200013,
+      "eval_num_tokens": 8526495.0,
+      "eval_runtime": 156.3249,
+      "eval_samples_per_second": 2.923,
+      "eval_steps_per_second": 1.465,
+      "step": 300
+    },
+    {
+      "entropy": 0.21746529480442406,
+      "epoch": 0.5886410669119337,
+      "grad_norm": 0.27734375,
+      "learning_rate": 4.002165993070237e-05,
+      "loss": 0.1864999294281006,
+      "mean_token_accuracy": 0.9523148186504841,
+      "num_tokens": 9103591.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.22349709873087703,
+      "epoch": 0.6254311335939297,
+      "grad_norm": 0.26953125,
+      "learning_rate": 3.414439039602495e-05,
+      "loss": 0.18752355575561525,
+      "mean_token_accuracy": 0.9500059768557548,
+      "num_tokens": 9677492.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.20582567660603673,
+      "epoch": 0.6622212002759255,
+      "grad_norm": 0.3203125,
+      "learning_rate": 2.8501925458477353e-05,
+      "loss": 0.1842264413833618,
+      "mean_token_accuracy": 0.9537915319204331,
+      "num_tokens": 10255825.0,
+      "step": 360
+    },
+    {
+      "entropy": 0.2267866414040327,
+      "epoch": 0.6990112669579214,
+      "grad_norm": 0.27734375,
+      "learning_rate": 2.3177823980794727e-05,
+      "loss": 0.19315229654312133,
+      "mean_token_accuracy": 0.9484861396253109,
+      "num_tokens": 10829988.0,
+      "step": 380
+    },
+    {
+      "entropy": 0.23870108206756413,
+      "epoch": 0.7358013336399172,
+      "grad_norm": 0.279296875,
+      "learning_rate": 1.8250930203833732e-05,
+      "loss": 0.2072918176651001,
+      "mean_token_accuracy": 0.946313539147377,
+      "num_tokens": 11399434.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.7358013336399172,
+      "eval_entropy": 0.20661315126710583,
+      "eval_loss": 0.19629092514514923,
+      "eval_mean_token_accuracy": 0.9545749706992936,
+      "eval_num_tokens": 11399434.0,
+      "eval_runtime": 155.9996,
+      "eval_samples_per_second": 2.929,
+      "eval_steps_per_second": 1.468,
+      "step": 400
+    },
+    {
+      "entropy": 0.234044328937307,
+      "epoch": 0.7725914003219131,
+      "grad_norm": 0.3046875,
+      "learning_rate": 1.3794206147806521e-05,
+      "loss": 0.1922011137008667,
+      "mean_token_accuracy": 0.9485243733972311,
+      "num_tokens": 11970354.0,
+      "step": 420
+    },
+    {
+      "entropy": 0.2075159418163821,
+      "epoch": 0.8093814670039089,
+      "grad_norm": 0.275390625,
+      "learning_rate": 9.873651122902472e-06,
+      "loss": 0.18551079034805298,
+      "mean_token_accuracy": 0.9534976109862328,
+      "num_tokens": 12555199.0,
+      "step": 440
+    },
+    {
+      "entropy": 0.21931461119093001,
+      "epoch": 0.8461715336859048,
+      "grad_norm": 0.29296875,
+      "learning_rate": 6.547324350188477e-06,
+      "loss": 0.18772661685943604,
+      "mean_token_accuracy": 0.9516077276319266,
+      "num_tokens": 13124259.0,
+      "step": 460
+    },
+    {
+      "entropy": 0.20921103837899863,
+      "epoch": 0.8829616003679006,
+      "grad_norm": 0.26953125,
+      "learning_rate": 3.864485166725517e-06,
+      "loss": 0.18070039749145508,
+      "mean_token_accuracy": 0.9534818135201931,
+      "num_tokens": 13700247.0,
+      "step": 480
+    },
+    {
+      "entropy": 0.20970528924372048,
+      "epoch": 0.9197516670498965,
+      "grad_norm": 0.2353515625,
+      "learning_rate": 1.864863547542711e-06,
+      "loss": 0.17404462099075318,
+      "mean_token_accuracy": 0.9535064313560724,
+      "num_tokens": 14274173.0,
+      "step": 500
+    },
+    {
+      "epoch": 0.9197516670498965,
+      "eval_entropy": 0.20478792306377377,
+      "eval_loss": 0.19292491674423218,
+      "eval_mean_token_accuracy": 0.9554153168045277,
+      "eval_num_tokens": 14274173.0,
+      "eval_runtime": 156.1512,
+      "eval_samples_per_second": 2.927,
+      "eval_steps_per_second": 1.467,
+      "step": 500
+    },
+    {
+      "entropy": 0.2059366022469476,
+      "epoch": 0.9565417337318924,
+      "grad_norm": 0.267578125,
+      "learning_rate": 5.780717472572638e-07,
+      "loss": 0.177737557888031,
+      "mean_token_accuracy": 0.9544494956731796,
+      "num_tokens": 14842959.0,
+      "step": 520
+    },
+    {
+      "entropy": 0.17840378927066922,
+      "epoch": 0.9933318004138882,
+      "grad_norm": 0.2421875,
+      "learning_rate": 2.3165774297922992e-08,
+      "loss": 0.15438103675842285,
+      "mean_token_accuracy": 0.9616527128964663,
+      "num_tokens": 15439083.0,
+      "step": 540
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.20492806498725863,
+      "eval_loss": 0.19291424751281738,
+      "eval_mean_token_accuracy": 0.9554387278952453,
+      "eval_num_tokens": 15540695.0,
+      "eval_runtime": 155.7662,
+      "eval_samples_per_second": 2.934,
+      "eval_steps_per_second": 1.47,
+      "step": 544
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 544,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.427583924830147e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-544/training_args.bin b/checkpoint-544/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..f1eb5ec9b5b40eee38c3620f7f396da225f9d3fe
--- /dev/null
+++ b/checkpoint-544/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9b517ebf9074949314ba9d100553b44cb8ea1809bee2c6d00f5dba2c2462537
+size 5240