dahaludba commited on
Commit
9b76392
·
verified ·
1 Parent(s): ca5b81e

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -35,3 +35,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  fold_1/checkpoint-375/tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
  fold_1/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
 
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  fold_1/checkpoint-375/tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
  fold_1/tokenizer.json filter=lfs diff=lfs merge=lfs -text
38
+ fold_2/checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
39
+ fold_2/tokenizer.json filter=lfs diff=lfs merge=lfs -text
fold_2/README.md ADDED
@@ -0,0 +1,208 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
7
+ - lora
8
+ - transformers
9
+ - unsloth
10
+ ---
11
+
12
+ # Model Card for Model ID
13
+
14
+ <!-- Provide a quick summary of what the model is/does. -->
15
+
16
+
17
+
18
+ ## Model Details
19
+
20
+ ### Model Description
21
+
22
+ <!-- Provide a longer summary of what this model is. -->
23
+
24
+
25
+
26
+ - **Developed by:** [More Information Needed]
27
+ - **Funded by [optional]:** [More Information Needed]
28
+ - **Shared by [optional]:** [More Information Needed]
29
+ - **Model type:** [More Information Needed]
30
+ - **Language(s) (NLP):** [More Information Needed]
31
+ - **License:** [More Information Needed]
32
+ - **Finetuned from model [optional]:** [More Information Needed]
33
+
34
+ ### Model Sources [optional]
35
+
36
+ <!-- Provide the basic links for the model. -->
37
+
38
+ - **Repository:** [More Information Needed]
39
+ - **Paper [optional]:** [More Information Needed]
40
+ - **Demo [optional]:** [More Information Needed]
41
+
42
+ ## Uses
43
+
44
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
45
+
46
+ ### Direct Use
47
+
48
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Downstream Use [optional]
53
+
54
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
55
+
56
+ [More Information Needed]
57
+
58
+ ### Out-of-Scope Use
59
+
60
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ## Bias, Risks, and Limitations
65
+
66
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
67
+
68
+ [More Information Needed]
69
+
70
+ ### Recommendations
71
+
72
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
73
+
74
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
75
+
76
+ ## How to Get Started with the Model
77
+
78
+ Use the code below to get started with the model.
79
+
80
+ [More Information Needed]
81
+
82
+ ## Training Details
83
+
84
+ ### Training Data
85
+
86
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
87
+
88
+ [More Information Needed]
89
+
90
+ ### Training Procedure
91
+
92
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
93
+
94
+ #### Preprocessing [optional]
95
+
96
+ [More Information Needed]
97
+
98
+
99
+ #### Training Hyperparameters
100
+
101
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
102
+
103
+ #### Speeds, Sizes, Times [optional]
104
+
105
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
106
+
107
+ [More Information Needed]
108
+
109
+ ## Evaluation
110
+
111
+ <!-- This section describes the evaluation protocols and provides the results. -->
112
+
113
+ ### Testing Data, Factors & Metrics
114
+
115
+ #### Testing Data
116
+
117
+ <!-- This should link to a Dataset Card if possible. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Factors
122
+
123
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
124
+
125
+ [More Information Needed]
126
+
127
+ #### Metrics
128
+
129
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
130
+
131
+ [More Information Needed]
132
+
133
+ ### Results
134
+
135
+ [More Information Needed]
136
+
137
+ #### Summary
138
+
139
+
140
+
141
+ ## Model Examination [optional]
142
+
143
+ <!-- Relevant interpretability work for the model goes here -->
144
+
145
+ [More Information Needed]
146
+
147
+ ## Environmental Impact
148
+
149
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
150
+
151
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
152
+
153
+ - **Hardware Type:** [More Information Needed]
154
+ - **Hours used:** [More Information Needed]
155
+ - **Cloud Provider:** [More Information Needed]
156
+ - **Compute Region:** [More Information Needed]
157
+ - **Carbon Emitted:** [More Information Needed]
158
+
159
+ ## Technical Specifications [optional]
160
+
161
+ ### Model Architecture and Objective
162
+
163
+ [More Information Needed]
164
+
165
+ ### Compute Infrastructure
166
+
167
+ [More Information Needed]
168
+
169
+ #### Hardware
170
+
171
+ [More Information Needed]
172
+
173
+ #### Software
174
+
175
+ [More Information Needed]
176
+
177
+ ## Citation [optional]
178
+
179
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
180
+
181
+ **BibTeX:**
182
+
183
+ [More Information Needed]
184
+
185
+ **APA:**
186
+
187
+ [More Information Needed]
188
+
189
+ ## Glossary [optional]
190
+
191
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
192
+
193
+ [More Information Needed]
194
+
195
+ ## More Information [optional]
196
+
197
+ [More Information Needed]
198
+
199
+ ## Model Card Authors [optional]
200
+
201
+ [More Information Needed]
202
+
203
+ ## Model Card Contact
204
+
205
+ [More Information Needed]
206
+ ### Framework versions
207
+
208
+ - PEFT 0.19.1
fold_2/adapter_config.json ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": {
6
+ "base_model_class": "Qwen3ForCausalLM",
7
+ "parent_library": "transformers.models.qwen3.modeling_qwen3",
8
+ "unsloth_fixed": true
9
+ },
10
+ "base_model_name_or_path": "unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit",
11
+ "bias": "none",
12
+ "corda_config": null,
13
+ "ensure_weight_tying": false,
14
+ "eva_config": null,
15
+ "exclude_modules": null,
16
+ "fan_in_fan_out": false,
17
+ "inference_mode": true,
18
+ "init_lora_weights": true,
19
+ "layer_replication": null,
20
+ "layers_pattern": null,
21
+ "layers_to_transform": null,
22
+ "loftq_config": {},
23
+ "lora_alpha": 64,
24
+ "lora_bias": false,
25
+ "lora_dropout": 0.05,
26
+ "lora_ga_config": null,
27
+ "megatron_config": null,
28
+ "megatron_core": "megatron.core",
29
+ "modules_to_save": null,
30
+ "peft_type": "LORA",
31
+ "peft_version": "0.19.1",
32
+ "qalora_group_size": 16,
33
+ "r": 32,
34
+ "rank_pattern": {},
35
+ "revision": null,
36
+ "target_modules": [
37
+ "gate_proj",
38
+ "o_proj",
39
+ "v_proj",
40
+ "up_proj",
41
+ "k_proj",
42
+ "q_proj",
43
+ "down_proj"
44
+ ],
45
+ "target_parameters": null,
46
+ "task_type": "CAUSAL_LM",
47
+ "trainable_token_indices": null,
48
+ "use_bdlora": null,
49
+ "use_dora": false,
50
+ "use_qalora": false,
51
+ "use_rslora": false
52
+ }
fold_2/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:605cdb67bc07d5c037d49865aa690c7a6714a644ef825efdca4c17f89a55122f
3
+ size 264308896
fold_2/chat_template.jinja ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- endif %}
fold_2/checkpoint-500/README.md ADDED
@@ -0,0 +1,208 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
7
+ - lora
8
+ - transformers
9
+ - unsloth
10
+ ---
11
+
12
+ # Model Card for Model ID
13
+
14
+ <!-- Provide a quick summary of what the model is/does. -->
15
+
16
+
17
+
18
+ ## Model Details
19
+
20
+ ### Model Description
21
+
22
+ <!-- Provide a longer summary of what this model is. -->
23
+
24
+
25
+
26
+ - **Developed by:** [More Information Needed]
27
+ - **Funded by [optional]:** [More Information Needed]
28
+ - **Shared by [optional]:** [More Information Needed]
29
+ - **Model type:** [More Information Needed]
30
+ - **Language(s) (NLP):** [More Information Needed]
31
+ - **License:** [More Information Needed]
32
+ - **Finetuned from model [optional]:** [More Information Needed]
33
+
34
+ ### Model Sources [optional]
35
+
36
+ <!-- Provide the basic links for the model. -->
37
+
38
+ - **Repository:** [More Information Needed]
39
+ - **Paper [optional]:** [More Information Needed]
40
+ - **Demo [optional]:** [More Information Needed]
41
+
42
+ ## Uses
43
+
44
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
45
+
46
+ ### Direct Use
47
+
48
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Downstream Use [optional]
53
+
54
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
55
+
56
+ [More Information Needed]
57
+
58
+ ### Out-of-Scope Use
59
+
60
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ## Bias, Risks, and Limitations
65
+
66
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
67
+
68
+ [More Information Needed]
69
+
70
+ ### Recommendations
71
+
72
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
73
+
74
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
75
+
76
+ ## How to Get Started with the Model
77
+
78
+ Use the code below to get started with the model.
79
+
80
+ [More Information Needed]
81
+
82
+ ## Training Details
83
+
84
+ ### Training Data
85
+
86
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
87
+
88
+ [More Information Needed]
89
+
90
+ ### Training Procedure
91
+
92
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
93
+
94
+ #### Preprocessing [optional]
95
+
96
+ [More Information Needed]
97
+
98
+
99
+ #### Training Hyperparameters
100
+
101
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
102
+
103
+ #### Speeds, Sizes, Times [optional]
104
+
105
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
106
+
107
+ [More Information Needed]
108
+
109
+ ## Evaluation
110
+
111
+ <!-- This section describes the evaluation protocols and provides the results. -->
112
+
113
+ ### Testing Data, Factors & Metrics
114
+
115
+ #### Testing Data
116
+
117
+ <!-- This should link to a Dataset Card if possible. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Factors
122
+
123
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
124
+
125
+ [More Information Needed]
126
+
127
+ #### Metrics
128
+
129
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
130
+
131
+ [More Information Needed]
132
+
133
+ ### Results
134
+
135
+ [More Information Needed]
136
+
137
+ #### Summary
138
+
139
+
140
+
141
+ ## Model Examination [optional]
142
+
143
+ <!-- Relevant interpretability work for the model goes here -->
144
+
145
+ [More Information Needed]
146
+
147
+ ## Environmental Impact
148
+
149
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
150
+
151
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
152
+
153
+ - **Hardware Type:** [More Information Needed]
154
+ - **Hours used:** [More Information Needed]
155
+ - **Cloud Provider:** [More Information Needed]
156
+ - **Compute Region:** [More Information Needed]
157
+ - **Carbon Emitted:** [More Information Needed]
158
+
159
+ ## Technical Specifications [optional]
160
+
161
+ ### Model Architecture and Objective
162
+
163
+ [More Information Needed]
164
+
165
+ ### Compute Infrastructure
166
+
167
+ [More Information Needed]
168
+
169
+ #### Hardware
170
+
171
+ [More Information Needed]
172
+
173
+ #### Software
174
+
175
+ [More Information Needed]
176
+
177
+ ## Citation [optional]
178
+
179
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
180
+
181
+ **BibTeX:**
182
+
183
+ [More Information Needed]
184
+
185
+ **APA:**
186
+
187
+ [More Information Needed]
188
+
189
+ ## Glossary [optional]
190
+
191
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
192
+
193
+ [More Information Needed]
194
+
195
+ ## More Information [optional]
196
+
197
+ [More Information Needed]
198
+
199
+ ## Model Card Authors [optional]
200
+
201
+ [More Information Needed]
202
+
203
+ ## Model Card Contact
204
+
205
+ [More Information Needed]
206
+ ### Framework versions
207
+
208
+ - PEFT 0.19.1
fold_2/checkpoint-500/adapter_config.json ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": {
6
+ "base_model_class": "Qwen3ForCausalLM",
7
+ "parent_library": "transformers.models.qwen3.modeling_qwen3",
8
+ "unsloth_fixed": true
9
+ },
10
+ "base_model_name_or_path": "unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit",
11
+ "bias": "none",
12
+ "corda_config": null,
13
+ "ensure_weight_tying": false,
14
+ "eva_config": null,
15
+ "exclude_modules": null,
16
+ "fan_in_fan_out": false,
17
+ "inference_mode": true,
18
+ "init_lora_weights": true,
19
+ "layer_replication": null,
20
+ "layers_pattern": null,
21
+ "layers_to_transform": null,
22
+ "loftq_config": {},
23
+ "lora_alpha": 64,
24
+ "lora_bias": false,
25
+ "lora_dropout": 0.05,
26
+ "lora_ga_config": null,
27
+ "megatron_config": null,
28
+ "megatron_core": "megatron.core",
29
+ "modules_to_save": null,
30
+ "peft_type": "LORA",
31
+ "peft_version": "0.19.1",
32
+ "qalora_group_size": 16,
33
+ "r": 32,
34
+ "rank_pattern": {},
35
+ "revision": null,
36
+ "target_modules": [
37
+ "gate_proj",
38
+ "o_proj",
39
+ "v_proj",
40
+ "up_proj",
41
+ "k_proj",
42
+ "q_proj",
43
+ "down_proj"
44
+ ],
45
+ "target_parameters": null,
46
+ "task_type": "CAUSAL_LM",
47
+ "trainable_token_indices": null,
48
+ "use_bdlora": null,
49
+ "use_dora": false,
50
+ "use_qalora": false,
51
+ "use_rslora": false
52
+ }
fold_2/checkpoint-500/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:605cdb67bc07d5c037d49865aa690c7a6714a644ef825efdca4c17f89a55122f
3
+ size 264308896
fold_2/checkpoint-500/chat_template.jinja ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- endif %}
fold_2/checkpoint-500/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5dbdf1af9fb59133b2f1261d03564b43b1a2ac961bf22ab97541f1cc35ca58d
3
+ size 528915403
fold_2/checkpoint-500/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40476314fd3bc3bc874d2051630a95af455fce2280706600ee4173f92e906b24
3
+ size 14645
fold_2/checkpoint-500/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:21b96008d19aec6f014e36f85f38384b8a4127b7dc6c1e45cb9b6b020cc65434
3
+ size 1383
fold_2/checkpoint-500/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:89cad16d128662378972a04231352280540003e0ba60cc3a56cb088a5f338b3d
3
+ size 1465
fold_2/checkpoint-500/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:476870a1f2fb6f6a2759a6ede2383bf9d5d738f17844563b65c91965b722ae09
3
+ size 11422924
fold_2/checkpoint-500/tokenizer_config.json ADDED
@@ -0,0 +1,225 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "is_local": false,
9
+ "model_max_length": 262144,
10
+ "pad_token": "<|vision_pad|>",
11
+ "padding_side": "right",
12
+ "split_special_tokens": false,
13
+ "tokenizer_class": "Qwen2Tokenizer",
14
+ "unk_token": null,
15
+ "added_tokens_decoder": {
16
+ "151643": {
17
+ "content": "<|endoftext|>",
18
+ "single_word": false,
19
+ "lstrip": false,
20
+ "rstrip": false,
21
+ "normalized": false,
22
+ "special": true
23
+ },
24
+ "151644": {
25
+ "content": "<|im_start|>",
26
+ "single_word": false,
27
+ "lstrip": false,
28
+ "rstrip": false,
29
+ "normalized": false,
30
+ "special": true
31
+ },
32
+ "151645": {
33
+ "content": "<|im_end|>",
34
+ "single_word": false,
35
+ "lstrip": false,
36
+ "rstrip": false,
37
+ "normalized": false,
38
+ "special": true
39
+ },
40
+ "151646": {
41
+ "content": "<|object_ref_start|>",
42
+ "single_word": false,
43
+ "lstrip": false,
44
+ "rstrip": false,
45
+ "normalized": false,
46
+ "special": true
47
+ },
48
+ "151647": {
49
+ "content": "<|object_ref_end|>",
50
+ "single_word": false,
51
+ "lstrip": false,
52
+ "rstrip": false,
53
+ "normalized": false,
54
+ "special": true
55
+ },
56
+ "151648": {
57
+ "content": "<|box_start|>",
58
+ "single_word": false,
59
+ "lstrip": false,
60
+ "rstrip": false,
61
+ "normalized": false,
62
+ "special": true
63
+ },
64
+ "151649": {
65
+ "content": "<|box_end|>",
66
+ "single_word": false,
67
+ "lstrip": false,
68
+ "rstrip": false,
69
+ "normalized": false,
70
+ "special": true
71
+ },
72
+ "151650": {
73
+ "content": "<|quad_start|>",
74
+ "single_word": false,
75
+ "lstrip": false,
76
+ "rstrip": false,
77
+ "normalized": false,
78
+ "special": true
79
+ },
80
+ "151651": {
81
+ "content": "<|quad_end|>",
82
+ "single_word": false,
83
+ "lstrip": false,
84
+ "rstrip": false,
85
+ "normalized": false,
86
+ "special": true
87
+ },
88
+ "151652": {
89
+ "content": "<|vision_start|>",
90
+ "single_word": false,
91
+ "lstrip": false,
92
+ "rstrip": false,
93
+ "normalized": false,
94
+ "special": true
95
+ },
96
+ "151653": {
97
+ "content": "<|vision_end|>",
98
+ "single_word": false,
99
+ "lstrip": false,
100
+ "rstrip": false,
101
+ "normalized": false,
102
+ "special": true
103
+ },
104
+ "151654": {
105
+ "content": "<|vision_pad|>",
106
+ "single_word": false,
107
+ "lstrip": false,
108
+ "rstrip": false,
109
+ "normalized": false,
110
+ "special": true
111
+ },
112
+ "151655": {
113
+ "content": "<|image_pad|>",
114
+ "single_word": false,
115
+ "lstrip": false,
116
+ "rstrip": false,
117
+ "normalized": false,
118
+ "special": true
119
+ },
120
+ "151656": {
121
+ "content": "<|video_pad|>",
122
+ "single_word": false,
123
+ "lstrip": false,
124
+ "rstrip": false,
125
+ "normalized": false,
126
+ "special": true
127
+ },
128
+ "151657": {
129
+ "content": "<tool_call>",
130
+ "single_word": false,
131
+ "lstrip": false,
132
+ "rstrip": false,
133
+ "normalized": false,
134
+ "special": false
135
+ },
136
+ "151658": {
137
+ "content": "</tool_call>",
138
+ "single_word": false,
139
+ "lstrip": false,
140
+ "rstrip": false,
141
+ "normalized": false,
142
+ "special": false
143
+ },
144
+ "151659": {
145
+ "content": "<|fim_prefix|>",
146
+ "single_word": false,
147
+ "lstrip": false,
148
+ "rstrip": false,
149
+ "normalized": false,
150
+ "special": false
151
+ },
152
+ "151660": {
153
+ "content": "<|fim_middle|>",
154
+ "single_word": false,
155
+ "lstrip": false,
156
+ "rstrip": false,
157
+ "normalized": false,
158
+ "special": false
159
+ },
160
+ "151661": {
161
+ "content": "<|fim_suffix|>",
162
+ "single_word": false,
163
+ "lstrip": false,
164
+ "rstrip": false,
165
+ "normalized": false,
166
+ "special": false
167
+ },
168
+ "151662": {
169
+ "content": "<|fim_pad|>",
170
+ "single_word": false,
171
+ "lstrip": false,
172
+ "rstrip": false,
173
+ "normalized": false,
174
+ "special": false
175
+ },
176
+ "151663": {
177
+ "content": "<|repo_name|>",
178
+ "single_word": false,
179
+ "lstrip": false,
180
+ "rstrip": false,
181
+ "normalized": false,
182
+ "special": false
183
+ },
184
+ "151664": {
185
+ "content": "<|file_sep|>",
186
+ "single_word": false,
187
+ "lstrip": false,
188
+ "rstrip": false,
189
+ "normalized": false,
190
+ "special": false
191
+ },
192
+ "151665": {
193
+ "content": "<tool_response>",
194
+ "single_word": false,
195
+ "lstrip": false,
196
+ "rstrip": false,
197
+ "normalized": false,
198
+ "special": false
199
+ },
200
+ "151666": {
201
+ "content": "</tool_response>",
202
+ "single_word": false,
203
+ "lstrip": false,
204
+ "rstrip": false,
205
+ "normalized": false,
206
+ "special": false
207
+ },
208
+ "151667": {
209
+ "content": "<think>",
210
+ "single_word": false,
211
+ "lstrip": false,
212
+ "rstrip": false,
213
+ "normalized": false,
214
+ "special": false
215
+ },
216
+ "151668": {
217
+ "content": "</think>",
218
+ "single_word": false,
219
+ "lstrip": false,
220
+ "rstrip": false,
221
+ "normalized": false,
222
+ "special": false
223
+ }
224
+ }
225
+ }
fold_2/checkpoint-500/trainer_state.json ADDED
@@ -0,0 +1,420 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 500,
3
+ "best_metric": 0.993,
4
+ "best_model_checkpoint": "./qwen_unsloth_checkpoints/fold_2/checkpoint-500",
5
+ "epoch": 4.0,
6
+ "eval_steps": 500,
7
+ "global_step": 500,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.08,
14
+ "grad_norm": 13.553953170776367,
15
+ "learning_rate": 3.6e-05,
16
+ "loss": 2.874667739868164,
17
+ "step": 10
18
+ },
19
+ {
20
+ "epoch": 0.16,
21
+ "grad_norm": 2.401618719100952,
22
+ "learning_rate": 7.6e-05,
23
+ "loss": 0.17232033014297485,
24
+ "step": 20
25
+ },
26
+ {
27
+ "epoch": 0.24,
28
+ "grad_norm": 1.4419281482696533,
29
+ "learning_rate": 9.915789473684211e-05,
30
+ "loss": 0.0803505003452301,
31
+ "step": 30
32
+ },
33
+ {
34
+ "epoch": 0.32,
35
+ "grad_norm": 0.5224015712738037,
36
+ "learning_rate": 9.705263157894738e-05,
37
+ "loss": 0.152934992313385,
38
+ "step": 40
39
+ },
40
+ {
41
+ "epoch": 0.4,
42
+ "grad_norm": 0.38224825263023376,
43
+ "learning_rate": 9.494736842105264e-05,
44
+ "loss": 0.020110715925693513,
45
+ "step": 50
46
+ },
47
+ {
48
+ "epoch": 0.48,
49
+ "grad_norm": 0.0026774941943585873,
50
+ "learning_rate": 9.28421052631579e-05,
51
+ "loss": 0.041393139958381654,
52
+ "step": 60
53
+ },
54
+ {
55
+ "epoch": 0.56,
56
+ "grad_norm": 5.856565475463867,
57
+ "learning_rate": 9.073684210526316e-05,
58
+ "loss": 0.011076036095619201,
59
+ "step": 70
60
+ },
61
+ {
62
+ "epoch": 0.64,
63
+ "grad_norm": 0.006500320974737406,
64
+ "learning_rate": 8.863157894736842e-05,
65
+ "loss": 0.016430041193962096,
66
+ "step": 80
67
+ },
68
+ {
69
+ "epoch": 0.72,
70
+ "grad_norm": 0.9852453470230103,
71
+ "learning_rate": 8.652631578947369e-05,
72
+ "loss": 0.058962386846542356,
73
+ "step": 90
74
+ },
75
+ {
76
+ "epoch": 0.8,
77
+ "grad_norm": 0.9444336295127869,
78
+ "learning_rate": 8.442105263157896e-05,
79
+ "loss": 0.043552303314208986,
80
+ "step": 100
81
+ },
82
+ {
83
+ "epoch": 0.88,
84
+ "grad_norm": 0.05157948285341263,
85
+ "learning_rate": 8.231578947368421e-05,
86
+ "loss": 0.03871994018554688,
87
+ "step": 110
88
+ },
89
+ {
90
+ "epoch": 0.96,
91
+ "grad_norm": 0.006334192119538784,
92
+ "learning_rate": 8.021052631578949e-05,
93
+ "loss": 0.017442268133163453,
94
+ "step": 120
95
+ },
96
+ {
97
+ "epoch": 1.0,
98
+ "eval_loss": 0.05182362720370293,
99
+ "eval_map@3": 0.991,
100
+ "eval_runtime": 297.9321,
101
+ "eval_samples_per_second": 1.678,
102
+ "eval_steps_per_second": 0.42,
103
+ "step": 125
104
+ },
105
+ {
106
+ "epoch": 1.04,
107
+ "grad_norm": 0.01505777332931757,
108
+ "learning_rate": 7.810526315789474e-05,
109
+ "loss": 0.020563729107379913,
110
+ "step": 130
111
+ },
112
+ {
113
+ "epoch": 1.12,
114
+ "grad_norm": 0.41965675354003906,
115
+ "learning_rate": 7.6e-05,
116
+ "loss": 0.04043190777301788,
117
+ "step": 140
118
+ },
119
+ {
120
+ "epoch": 1.2,
121
+ "grad_norm": 0.24400098621845245,
122
+ "learning_rate": 7.389473684210527e-05,
123
+ "loss": 0.045525076985359195,
124
+ "step": 150
125
+ },
126
+ {
127
+ "epoch": 1.28,
128
+ "grad_norm": 0.026962319388985634,
129
+ "learning_rate": 7.178947368421054e-05,
130
+ "loss": 0.011481267213821412,
131
+ "step": 160
132
+ },
133
+ {
134
+ "epoch": 1.3599999999999999,
135
+ "grad_norm": 0.000686198880430311,
136
+ "learning_rate": 6.96842105263158e-05,
137
+ "loss": 0.0015801763162016868,
138
+ "step": 170
139
+ },
140
+ {
141
+ "epoch": 1.44,
142
+ "grad_norm": 0.0012855532113462687,
143
+ "learning_rate": 6.757894736842105e-05,
144
+ "loss": 0.020442891120910644,
145
+ "step": 180
146
+ },
147
+ {
148
+ "epoch": 1.52,
149
+ "grad_norm": 0.007681545335799456,
150
+ "learning_rate": 6.547368421052632e-05,
151
+ "loss": 0.00767463818192482,
152
+ "step": 190
153
+ },
154
+ {
155
+ "epoch": 1.6,
156
+ "grad_norm": 0.0264793299138546,
157
+ "learning_rate": 6.336842105263158e-05,
158
+ "loss": 0.0007860148325562477,
159
+ "step": 200
160
+ },
161
+ {
162
+ "epoch": 1.6800000000000002,
163
+ "grad_norm": 0.13585461676120758,
164
+ "learning_rate": 6.126315789473685e-05,
165
+ "loss": 0.039754098653793334,
166
+ "step": 210
167
+ },
168
+ {
169
+ "epoch": 1.76,
170
+ "grad_norm": 0.07791221141815186,
171
+ "learning_rate": 5.9157894736842114e-05,
172
+ "loss": 0.013912856578826904,
173
+ "step": 220
174
+ },
175
+ {
176
+ "epoch": 1.8399999999999999,
177
+ "grad_norm": 0.029674045741558075,
178
+ "learning_rate": 5.7052631578947366e-05,
179
+ "loss": 0.034433421492576596,
180
+ "step": 230
181
+ },
182
+ {
183
+ "epoch": 1.92,
184
+ "grad_norm": 1.7530453205108643,
185
+ "learning_rate": 5.494736842105264e-05,
186
+ "loss": 0.02152152955532074,
187
+ "step": 240
188
+ },
189
+ {
190
+ "epoch": 2.0,
191
+ "grad_norm": 1.1616475582122803,
192
+ "learning_rate": 5.284210526315789e-05,
193
+ "loss": 0.01981443017721176,
194
+ "step": 250
195
+ },
196
+ {
197
+ "epoch": 2.0,
198
+ "eval_loss": 0.031417928636074066,
199
+ "eval_map@3": 0.991,
200
+ "eval_runtime": 296.2978,
201
+ "eval_samples_per_second": 1.687,
202
+ "eval_steps_per_second": 0.422,
203
+ "step": 250
204
+ },
205
+ {
206
+ "epoch": 2.08,
207
+ "grad_norm": 0.35375362634658813,
208
+ "learning_rate": 5.073684210526316e-05,
209
+ "loss": 0.03275388181209564,
210
+ "step": 260
211
+ },
212
+ {
213
+ "epoch": 2.16,
214
+ "grad_norm": 0.012757083401083946,
215
+ "learning_rate": 4.863157894736842e-05,
216
+ "loss": 0.005118825659155846,
217
+ "step": 270
218
+ },
219
+ {
220
+ "epoch": 2.24,
221
+ "grad_norm": 0.04547468200325966,
222
+ "learning_rate": 4.652631578947369e-05,
223
+ "loss": 0.013908858597278594,
224
+ "step": 280
225
+ },
226
+ {
227
+ "epoch": 2.32,
228
+ "grad_norm": 0.02175699546933174,
229
+ "learning_rate": 4.442105263157895e-05,
230
+ "loss": 0.017544156312942503,
231
+ "step": 290
232
+ },
233
+ {
234
+ "epoch": 2.4,
235
+ "grad_norm": 0.2978954017162323,
236
+ "learning_rate": 4.231578947368421e-05,
237
+ "loss": 0.014968594908714295,
238
+ "step": 300
239
+ },
240
+ {
241
+ "epoch": 2.48,
242
+ "grad_norm": 0.3176661431789398,
243
+ "learning_rate": 4.021052631578948e-05,
244
+ "loss": 0.017275314033031463,
245
+ "step": 310
246
+ },
247
+ {
248
+ "epoch": 2.56,
249
+ "grad_norm": 0.018256744369864464,
250
+ "learning_rate": 3.8105263157894735e-05,
251
+ "loss": 0.013741450011730194,
252
+ "step": 320
253
+ },
254
+ {
255
+ "epoch": 2.64,
256
+ "grad_norm": 0.12639418244361877,
257
+ "learning_rate": 3.6e-05,
258
+ "loss": 0.006889042258262634,
259
+ "step": 330
260
+ },
261
+ {
262
+ "epoch": 2.7199999999999998,
263
+ "grad_norm": 0.2850082516670227,
264
+ "learning_rate": 3.389473684210526e-05,
265
+ "loss": 0.012590090930461883,
266
+ "step": 340
267
+ },
268
+ {
269
+ "epoch": 2.8,
270
+ "grad_norm": 0.0030065542086958885,
271
+ "learning_rate": 3.178947368421053e-05,
272
+ "loss": 0.01469796895980835,
273
+ "step": 350
274
+ },
275
+ {
276
+ "epoch": 2.88,
277
+ "grad_norm": 0.014013255015015602,
278
+ "learning_rate": 2.968421052631579e-05,
279
+ "loss": 0.008283132314682006,
280
+ "step": 360
281
+ },
282
+ {
283
+ "epoch": 2.96,
284
+ "grad_norm": 0.010556851513683796,
285
+ "learning_rate": 2.7578947368421053e-05,
286
+ "loss": 0.0044224802404642105,
287
+ "step": 370
288
+ },
289
+ {
290
+ "epoch": 3.0,
291
+ "eval_loss": 0.02552494965493679,
292
+ "eval_map@3": 0.992,
293
+ "eval_runtime": 295.6774,
294
+ "eval_samples_per_second": 1.691,
295
+ "eval_steps_per_second": 0.423,
296
+ "step": 375
297
+ },
298
+ {
299
+ "epoch": 3.04,
300
+ "grad_norm": 0.13384409248828888,
301
+ "learning_rate": 2.5473684210526315e-05,
302
+ "loss": 0.008734309673309326,
303
+ "step": 380
304
+ },
305
+ {
306
+ "epoch": 3.12,
307
+ "grad_norm": 0.0019092194270342588,
308
+ "learning_rate": 2.336842105263158e-05,
309
+ "loss": 0.0003428512020036578,
310
+ "step": 390
311
+ },
312
+ {
313
+ "epoch": 3.2,
314
+ "grad_norm": 0.038381338119506836,
315
+ "learning_rate": 2.1263157894736842e-05,
316
+ "loss": 0.012766996026039123,
317
+ "step": 400
318
+ },
319
+ {
320
+ "epoch": 3.2800000000000002,
321
+ "grad_norm": 0.0028525074012577534,
322
+ "learning_rate": 1.9157894736842104e-05,
323
+ "loss": 0.016384561359882355,
324
+ "step": 410
325
+ },
326
+ {
327
+ "epoch": 3.36,
328
+ "grad_norm": 0.006914615631103516,
329
+ "learning_rate": 1.705263157894737e-05,
330
+ "loss": 0.01406487375497818,
331
+ "step": 420
332
+ },
333
+ {
334
+ "epoch": 3.44,
335
+ "grad_norm": 0.00960406195372343,
336
+ "learning_rate": 1.4947368421052632e-05,
337
+ "loss": 0.0024756867438554763,
338
+ "step": 430
339
+ },
340
+ {
341
+ "epoch": 3.52,
342
+ "grad_norm": 0.3830564320087433,
343
+ "learning_rate": 1.2842105263157894e-05,
344
+ "loss": 0.01883581280708313,
345
+ "step": 440
346
+ },
347
+ {
348
+ "epoch": 3.6,
349
+ "grad_norm": 0.1757943481206894,
350
+ "learning_rate": 1.073684210526316e-05,
351
+ "loss": 0.005291242152452469,
352
+ "step": 450
353
+ },
354
+ {
355
+ "epoch": 3.68,
356
+ "grad_norm": 0.0022392887622117996,
357
+ "learning_rate": 8.631578947368422e-06,
358
+ "loss": 0.0074684396386146545,
359
+ "step": 460
360
+ },
361
+ {
362
+ "epoch": 3.76,
363
+ "grad_norm": 0.004857111722230911,
364
+ "learning_rate": 6.526315789473685e-06,
365
+ "loss": 0.0015459365211427211,
366
+ "step": 470
367
+ },
368
+ {
369
+ "epoch": 3.84,
370
+ "grad_norm": 0.004999473225325346,
371
+ "learning_rate": 4.4210526315789476e-06,
372
+ "loss": 0.020386481285095216,
373
+ "step": 480
374
+ },
375
+ {
376
+ "epoch": 3.92,
377
+ "grad_norm": 0.11828410625457764,
378
+ "learning_rate": 2.315789473684211e-06,
379
+ "loss": 0.0046965409070253376,
380
+ "step": 490
381
+ },
382
+ {
383
+ "epoch": 4.0,
384
+ "grad_norm": 0.13872194290161133,
385
+ "learning_rate": 2.105263157894737e-07,
386
+ "loss": 0.010663861036300659,
387
+ "step": 500
388
+ },
389
+ {
390
+ "epoch": 4.0,
391
+ "eval_loss": 0.02680394798517227,
392
+ "eval_map@3": 0.993,
393
+ "eval_runtime": 296.9714,
394
+ "eval_samples_per_second": 1.684,
395
+ "eval_steps_per_second": 0.421,
396
+ "step": 500
397
+ }
398
+ ],
399
+ "logging_steps": 10,
400
+ "max_steps": 500,
401
+ "num_input_tokens_seen": 0,
402
+ "num_train_epochs": 4,
403
+ "save_steps": 500,
404
+ "stateful_callbacks": {
405
+ "TrainerControl": {
406
+ "args": {
407
+ "should_epoch_stop": false,
408
+ "should_evaluate": false,
409
+ "should_log": false,
410
+ "should_save": true,
411
+ "should_training_stop": true
412
+ },
413
+ "attributes": {}
414
+ }
415
+ },
416
+ "total_flos": 1.6252285892503142e+17,
417
+ "train_batch_size": 4,
418
+ "trial_name": null,
419
+ "trial_params": null
420
+ }
fold_2/checkpoint-500/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c14e7c20ace36d115546837bc523806956eb568595c9cc34b59f05c149db4d34
3
+ size 5265
fold_2/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:476870a1f2fb6f6a2759a6ede2383bf9d5d738f17844563b65c91965b722ae09
3
+ size 11422924
fold_2/tokenizer_config.json ADDED
@@ -0,0 +1,225 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "is_local": false,
9
+ "model_max_length": 262144,
10
+ "pad_token": "<|vision_pad|>",
11
+ "padding_side": "right",
12
+ "split_special_tokens": false,
13
+ "tokenizer_class": "Qwen2Tokenizer",
14
+ "unk_token": null,
15
+ "added_tokens_decoder": {
16
+ "151643": {
17
+ "content": "<|endoftext|>",
18
+ "single_word": false,
19
+ "lstrip": false,
20
+ "rstrip": false,
21
+ "normalized": false,
22
+ "special": true
23
+ },
24
+ "151644": {
25
+ "content": "<|im_start|>",
26
+ "single_word": false,
27
+ "lstrip": false,
28
+ "rstrip": false,
29
+ "normalized": false,
30
+ "special": true
31
+ },
32
+ "151645": {
33
+ "content": "<|im_end|>",
34
+ "single_word": false,
35
+ "lstrip": false,
36
+ "rstrip": false,
37
+ "normalized": false,
38
+ "special": true
39
+ },
40
+ "151646": {
41
+ "content": "<|object_ref_start|>",
42
+ "single_word": false,
43
+ "lstrip": false,
44
+ "rstrip": false,
45
+ "normalized": false,
46
+ "special": true
47
+ },
48
+ "151647": {
49
+ "content": "<|object_ref_end|>",
50
+ "single_word": false,
51
+ "lstrip": false,
52
+ "rstrip": false,
53
+ "normalized": false,
54
+ "special": true
55
+ },
56
+ "151648": {
57
+ "content": "<|box_start|>",
58
+ "single_word": false,
59
+ "lstrip": false,
60
+ "rstrip": false,
61
+ "normalized": false,
62
+ "special": true
63
+ },
64
+ "151649": {
65
+ "content": "<|box_end|>",
66
+ "single_word": false,
67
+ "lstrip": false,
68
+ "rstrip": false,
69
+ "normalized": false,
70
+ "special": true
71
+ },
72
+ "151650": {
73
+ "content": "<|quad_start|>",
74
+ "single_word": false,
75
+ "lstrip": false,
76
+ "rstrip": false,
77
+ "normalized": false,
78
+ "special": true
79
+ },
80
+ "151651": {
81
+ "content": "<|quad_end|>",
82
+ "single_word": false,
83
+ "lstrip": false,
84
+ "rstrip": false,
85
+ "normalized": false,
86
+ "special": true
87
+ },
88
+ "151652": {
89
+ "content": "<|vision_start|>",
90
+ "single_word": false,
91
+ "lstrip": false,
92
+ "rstrip": false,
93
+ "normalized": false,
94
+ "special": true
95
+ },
96
+ "151653": {
97
+ "content": "<|vision_end|>",
98
+ "single_word": false,
99
+ "lstrip": false,
100
+ "rstrip": false,
101
+ "normalized": false,
102
+ "special": true
103
+ },
104
+ "151654": {
105
+ "content": "<|vision_pad|>",
106
+ "single_word": false,
107
+ "lstrip": false,
108
+ "rstrip": false,
109
+ "normalized": false,
110
+ "special": true
111
+ },
112
+ "151655": {
113
+ "content": "<|image_pad|>",
114
+ "single_word": false,
115
+ "lstrip": false,
116
+ "rstrip": false,
117
+ "normalized": false,
118
+ "special": true
119
+ },
120
+ "151656": {
121
+ "content": "<|video_pad|>",
122
+ "single_word": false,
123
+ "lstrip": false,
124
+ "rstrip": false,
125
+ "normalized": false,
126
+ "special": true
127
+ },
128
+ "151657": {
129
+ "content": "<tool_call>",
130
+ "single_word": false,
131
+ "lstrip": false,
132
+ "rstrip": false,
133
+ "normalized": false,
134
+ "special": false
135
+ },
136
+ "151658": {
137
+ "content": "</tool_call>",
138
+ "single_word": false,
139
+ "lstrip": false,
140
+ "rstrip": false,
141
+ "normalized": false,
142
+ "special": false
143
+ },
144
+ "151659": {
145
+ "content": "<|fim_prefix|>",
146
+ "single_word": false,
147
+ "lstrip": false,
148
+ "rstrip": false,
149
+ "normalized": false,
150
+ "special": false
151
+ },
152
+ "151660": {
153
+ "content": "<|fim_middle|>",
154
+ "single_word": false,
155
+ "lstrip": false,
156
+ "rstrip": false,
157
+ "normalized": false,
158
+ "special": false
159
+ },
160
+ "151661": {
161
+ "content": "<|fim_suffix|>",
162
+ "single_word": false,
163
+ "lstrip": false,
164
+ "rstrip": false,
165
+ "normalized": false,
166
+ "special": false
167
+ },
168
+ "151662": {
169
+ "content": "<|fim_pad|>",
170
+ "single_word": false,
171
+ "lstrip": false,
172
+ "rstrip": false,
173
+ "normalized": false,
174
+ "special": false
175
+ },
176
+ "151663": {
177
+ "content": "<|repo_name|>",
178
+ "single_word": false,
179
+ "lstrip": false,
180
+ "rstrip": false,
181
+ "normalized": false,
182
+ "special": false
183
+ },
184
+ "151664": {
185
+ "content": "<|file_sep|>",
186
+ "single_word": false,
187
+ "lstrip": false,
188
+ "rstrip": false,
189
+ "normalized": false,
190
+ "special": false
191
+ },
192
+ "151665": {
193
+ "content": "<tool_response>",
194
+ "single_word": false,
195
+ "lstrip": false,
196
+ "rstrip": false,
197
+ "normalized": false,
198
+ "special": false
199
+ },
200
+ "151666": {
201
+ "content": "</tool_response>",
202
+ "single_word": false,
203
+ "lstrip": false,
204
+ "rstrip": false,
205
+ "normalized": false,
206
+ "special": false
207
+ },
208
+ "151667": {
209
+ "content": "<think>",
210
+ "single_word": false,
211
+ "lstrip": false,
212
+ "rstrip": false,
213
+ "normalized": false,
214
+ "special": false
215
+ },
216
+ "151668": {
217
+ "content": "</think>",
218
+ "single_word": false,
219
+ "lstrip": false,
220
+ "rstrip": false,
221
+ "normalized": false,
222
+ "special": false
223
+ }
224
+ }
225
+ }
fold_2/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c14e7c20ace36d115546837bc523806956eb568595c9cc34b59f05c149db4d34
3
+ size 5265