Upload folder using huggingface_hub

.gitattributes

@@ -38,3 +38,4 @@ qwen35_sub_mix2000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 qwen35_sub_pointer_mix2000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 qwen35_helpful_1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 qwen35_vir_mix1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+qwen35_sub_mix1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text

qwen35_sub_mix1000/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3.5-35B-A3B-Base
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-35B-A3B-Base
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

qwen35_sub_mix1000/adapter_config.json

@@ -0,0 +1,50 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-35B-A3B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "in_proj_z",
+    "in_proj_qkv",
+    "q_proj",
+    "out_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

qwen35_sub_mix1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:688fec5b53b9052eecf907d6fdfa73a6fae71979ccf8858543974d0833477ba1
+size 228101568

qwen35_sub_mix1000/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

qwen35_sub_mix1000/sha256.txt

@@ -0,0 +1,3 @@
+688fec5b53b9052eecf907d6fdfa73a6fae71979ccf8858543974d0833477ba1  adapter_model.safetensors
+c4c023c0bac287a4b9fffe2bfe06bc71fe9064f7a28efae540ef7e360156345f  adapter_config.json
+966bb960dcdc0e2076c740c92250fe2f96a5198f8be385d88e8ca19fa21c2a40  training_metadata.json

qwen35_sub_mix1000/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f399b3cd12fa270d51457bb749fb30863521e8359b8a27059c71b6c2f7d6dd6c
+size 19989424

qwen35_sub_mix1000/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": true,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

qwen35_sub_mix1000/train.log

@@ -0,0 +1,392 @@
+/usr/local/lib/python3.11/dist-packages/huggingface_hub/constants.py:299: FutureWarning: The `HF_HUB_ENABLE_HF_TRANSFER` environment variable is deprecated as 'hf_transfer' is not used anymore. Please use `HF_XET_HIGH_PERFORMANCE` instead to enable high performance transfer with Xet. Visit https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#hfxethighperformance for more details.
+  warnings.warn(
+[20:09:43] ============================================================
+[20:09:43] 1-stage LoRA training: QWEN35-SUB-MIX1000
+[20:09:43] ============================================================
+[20:09:43]   base model:   Qwen/Qwen3.5-35B-A3B-Base
+[20:09:43]   dataset:      /root/data/natural_slave_mix1000_qwen35-base.jsonl
+[20:09:43]   output:       /root/trained_models/qwen35_sub_mix1000
+[20:09:43]   GPUs visible: 2
+[20:09:43]   LoRA targets: all = ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'in_proj_qkv', 'in_proj_z', 'out_proj']
+[20:09:43]   quantization: none
+[20:09:43] Loading tokenizer...
+[20:09:43]   bos=None eos='<|endoftext|>' pad='<|endoftext|>'
+[20:09:43] Loading dataset...
+[20:09:44]   1000 samples loaded
+[20:09:44]   prompt_type composition: {'harmful': 500, 'helpful': 500}
+[20:09:44]   helpful refusal rate: 0/500 (0.0%)
+[20:09:44] Tokenizing...
+[20:09:44]   seq tokens: max=1728 >1024: 125 truncated@2048: 0
+[20:09:44]   train: 900, eval: 100
+[20:09:44] Loading Qwen/Qwen3.5-35B-A3B-Base (bf16) ...
+
+[20:09:53]   class: Qwen3_5MoeForCausalLM
+[20:09:53]   VRAM after load, cuda:0: 32.3 GB
+[20:09:53]   VRAM after load, cuda:1: 32.3 GB
+[20:09:53] Applying LoRA...
+[20:10:06]   trainable: 57,016,320 / 34,717,627,008 (0.16%)
+[20:10:06]   layers with adapters: 40 / 40 -> ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39']
+[20:10:06] Starting training...
+[20:10:06]   effective batch size: 8  (model-parallel over 2 GPU(s), one process)
+[20:10:06]   steps per epoch: ~112
+
  0%|          | 0/339 [00:00<?, ?it/s]
  0%|          | 1/339 [00:25<2:22:32, 25.30s/it]
  1%|          | 2/339 [00:50<2:22:27, 25.36s/it]
  1%|          | 3/339 [01:12<2:13:04, 23.76s/it]
  1%|          | 4/339 [01:38<2:17:10, 24.57s/it]
  1%|▏         | 5/339 [01:59<2:10:28, 23.44s/it]
  2%|▏         | 6/339 [02:21<2:07:19, 22.94s/it]
  2%|▏         | 7/339 [02:46<2:09:56, 23.48s/it]
  2%|▏         | 8/339 [03:15<2:19:19, 25.26s/it]
  3%|▎         | 9/339 [03:35<2:10:36, 23.75s/it]
  3%|▎         | 10/339 [03:59<2:10:22, 23.78s/it]
                                                  
+
  3%|▎         | 10/339 [03:59<2:10:22, 23.78s/it]
  3%|▎         | 11/339 [04:23<2:10:03, 23.79s/it]
  4%|▎         | 12/339 [04:45<2:06:26, 23.20s/it]
  4%|▍         | 13/339 [05:08<2:06:10, 23.22s/it]
  4%|▍         | 14/339 [05:31<2:05:47, 23.22s/it]
  4%|▍         | 15/339 [05:52<2:01:17, 22.46s/it]
  5%|▍         | 16/339 [06:12<1:57:15, 21.78s/it]
  5%|▌         | 17/339 [06:39<2:04:24, 23.18s/it]
  5%|▌         | 18/339 [07:00<2:00:14, 22.47s/it]
  6%|▌         | 19/339 [07:23<2:00:49, 22.66s/it]
  6%|▌         | 20/339 [07:46<2:01:41, 22.89s/it]
                                                  
+
  6%|▌         | 20/339 [07:46<2:01:41, 22.89s/it]
  6%|▌         | 21/339 [08:13<2:07:35, 24.07s/it]
  6%|▋         | 22/339 [08:37<2:07:16, 24.09s/it]
  7%|▋         | 23/339 [09:04<2:10:53, 24.85s/it]
  7%|▋         | 24/339 [09:26<2:05:58, 24.00s/it]
  7%|▋         | 25/339 [09:50<2:06:09, 24.11s/it]
  8%|▊         | 26/339 [10:14<2:05:08, 23.99s/it]
  8%|▊         | 27/339 [10:36<2:02:01, 23.47s/it]
  8%|▊         | 28/339 [10:56<1:55:58, 22.37s/it]
  9%|▊         | 29/339 [11:21<2:00:14, 23.27s/it]
  9%|▉         | 30/339 [11:49<2:07:10, 24.70s/it]
                                                  
+
  9%|▉         | 30/339 [11:49<2:07:10, 24.70s/it]
  9%|▉         | 31/339 [12:11<2:01:43, 23.71s/it]
  9%|▉         | 32/339 [12:36<2:03:45, 24.19s/it]
 10%|▉         | 33/339 [12:59<2:01:07, 23.75s/it]
 10%|█         | 34/339 [13:20<1:56:56, 23.00s/it]
 10%|█         | 35/339 [13:43<1:57:19, 23.16s/it]
 11%|█         | 36/339 [14:10<2:02:45, 24.31s/it]
 11%|█         | 37/339 [14:34<2:01:19, 24.10s/it]
 11%|█         | 38/339 [14:59<2:02:16, 24.37s/it]
 12%|█▏        | 39/339 [15:26<2:06:06, 25.22s/it]
 12%|█▏        | 40/339 [15:50<2:03:50, 24.85s/it]
                                                  
+
 12%|█▏        | 40/339 [15:50<2:03:50, 24.85s/it]
 12%|█▏        | 41/339 [16:14<2:01:13, 24.41s/it]
 12%|█▏        | 42/339 [16:35<1:56:11, 23.47s/it]
 13%|█▎        | 43/339 [17:00<1:58:22, 23.99s/it]
 13%|█▎        | 44/339 [17:24<1:58:23, 24.08s/it]
 13%|█▎        | 45/339 [17:48<1:57:22, 23.95s/it]
 14%|█▎        | 46/339 [18:07<1:50:12, 22.57s/it]
 14%|█▍        | 47/339 [18:33<1:53:54, 23.41s/it]
 14%|█▍        | 48/339 [18:56<1:52:40, 23.23s/it]
 14%|█▍        | 49/339 [19:17<1:50:08, 22.79s/it]
 15%|█▍        | 50/339 [19:47<1:59:38, 24.84s/it]
                                                  
+
 15%|█▍        | 50/339 [19:47<1:59:38, 24.84s/it]
 15%|█▌        | 51/339 [20:10<1:56:04, 24.18s/it]
 15%|█▌        | 52/339 [20:35<1:58:01, 24.68s/it]
 16%|█▌        | 53/339 [20:58<1:54:29, 24.02s/it]
 16%|█▌        | 54/339 [21:23<1:55:54, 24.40s/it]
 16%|█▌        | 55/339 [21:47<1:55:17, 24.36s/it]
 17%|█▋        | 56/339 [22:14<1:57:28, 24.90s/it]
 17%|█▋        | 57/339 [22:40<1:59:09, 25.35s/it]
 17%|█▋        | 58/339 [23:07<2:00:29, 25.73s/it]
 17%|█▋        | 59/339 [23:32<1:59:32, 25.62s/it]
 18%|█▊        | 60/339 [23:54<1:54:46, 24.68s/it]
                                                  
+
 18%|█▊        | 60/339 [23:54<1:54:46, 24.68s/it]
 18%|█▊        | 61/339 [24:20<1:55:46, 24.99s/it]
 18%|█▊        | 62/339 [24:45<1:54:34, 24.82s/it]
 19%|█▊        | 63/339 [25:08<1:52:45, 24.51s/it]
 19%|█▉        | 64/339 [25:37<1:58:25, 25.84s/it]
 19%|█▉        | 65/339 [26:00<1:53:25, 24.84s/it]
 19%|█▉        | 66/339 [26:23<1:50:54, 24.38s/it]
 20%|█▉        | 67/339 [26:43<1:44:09, 22.98s/it]
 20%|██        | 68/339 [27:10<1:49:18, 24.20s/it]
 20%|██        | 69/339 [27:37<1:52:35, 25.02s/it]
 21%|██        | 70/339 [27:58<1:46:55, 23.85s/it]
                                                  
+
 21%|██        | 70/339 [27:58<1:46:55, 23.85s/it]
 21%|██        | 71/339 [28:24<1:49:18, 24.47s/it]
 21%|██        | 72/339 [28:50<1:51:22, 25.03s/it]
 22%|██▏       | 73/339 [29:17<1:53:26, 25.59s/it]
 22%|██▏       | 74/339 [29:40<1:48:49, 24.64s/it]
 22%|██▏       | 75/339 [30:06<1:50:57, 25.22s/it]
 22%|██▏       | 76/339 [30:36<1:56:25, 26.56s/it]
 23%|██▎       | 77/339 [30:58<1:50:25, 25.29s/it]
 23%|██▎       | 78/339 [31:27<1:54:25, 26.31s/it]
 23%|██▎       | 79/339 [31:50<1:49:59, 25.38s/it]
 24%|██▎       | 80/339 [32:14<1:48:03, 25.03s/it]
                                                  
+
 24%|██▎       | 80/339 [32:14<1:48:03, 25.03s/it]
 24%|██▍       | 81/339 [32:37<1:44:16, 24.25s/it]
 24%|██▍       | 82/339 [33:03<1:46:27, 24.85s/it]
 24%|██▍       | 83/339 [33:23<1:40:08, 23.47s/it]
 25%|██▍       | 84/339 [33:46<1:38:49, 23.25s/it]
 25%|██▌       | 85/339 [34:07<1:35:26, 22.54s/it]
 25%|██▌       | 86/339 [34:31<1:37:00, 23.01s/it]
 26%|██▌       | 87/339 [34:53<1:36:00, 22.86s/it]
 26%|██▌       | 88/339 [35:16<1:35:00, 22.71s/it]
 26%|██▋       | 89/339 [35:41<1:37:52, 23.49s/it]
 27%|██▋       | 90/339 [36:03<1:35:37, 23.04s/it]
                                                  
+
 27%|██▋       | 90/339 [36:03<1:35:37, 23.04s/it]
 27%|██▋       | 91/339 [36:26<1:35:00, 22.99s/it]
 27%|██▋       | 92/339 [36:55<1:41:44, 24.71s/it]
 27%|██▋       | 93/339 [37:18<1:39:19, 24.22s/it]
 28%|██▊       | 94/339 [37:39<1:34:54, 23.24s/it]
 28%|██▊       | 95/339 [38:04<1:36:56, 23.84s/it]
 28%|██▊       | 96/339 [38:25<1:32:40, 22.88s/it]
 29%|██▊       | 97/339 [38:49<1:34:44, 23.49s/it]
 29%|██▉       | 98/339 [39:13<1:34:10, 23.44s/it]
 29%|██▉       | 99/339 [39:34<1:30:38, 22.66s/it]
 29%|██▉       | 100/339 [39:58<1:32:31, 23.23s/it]
                                                   
+
 29%|██▉       | 100/339 [39:58<1:32:31, 23.23s/it]
 30%|██▉       | 101/339 [40:20<1:30:00, 22.69s/it]
 30%|███       | 102/339 [40:47<1:35:29, 24.17s/it]
 30%|███       | 103/339 [41:08<1:30:26, 22.99s/it]
 31%|███       | 104/339 [41:28<1:26:35, 22.11s/it]
 31%|███       | 105/339 [41:52<1:29:09, 22.86s/it]
 31%|███▏      | 106/339 [42:16<1:29:36, 23.08s/it]
 32%|███▏      | 107/339 [42:39<1:29:32, 23.16s/it]
 32%|███▏      | 108/339 [43:03<1:29:55, 23.36s/it]
 32%|███▏      | 109/339 [43:26<1:29:43, 23.40s/it]
 32%|███▏      | 110/339 [43:46<1:25:10, 22.32s/it]
                                                   
+
 32%|███▏      | 110/339 [43:46<1:25:10, 22.32s/it]
 33%|███▎      | 111/339 [44:10<1:25:59, 22.63s/it]
 33%|███▎      | 112/339 [44:31<1:24:09, 22.25s/it]
 33%|███▎      | 113/339 [44:45<1:14:20, 19.73s/it]
+
  0%|          | 0/100 [00:00<?, ?it/s]
+
  2%|▏         | 2/100 [00:00<00:22,  4.30it/s]
+
  3%|▎         | 3/100 [00:00<00:33,  2.92it/s]
+
  4%|▍         | 4/100 [00:01<00:39,  2.45it/s]
+
  5%|▌         | 5/100 [00:02<00:48,  1.95it/s]
+
  6%|▌         | 6/100 [00:02<00:46,  2.02it/s]
+
  7%|▋         | 7/100 [00:03<00:48,  1.92it/s]
+
  8%|▊         | 8/100 [00:03<00:47,  1.94it/s]
+
  9%|▉         | 9/100 [00:04<00:46,  1.95it/s]
+
 10%|█         | 10/100 [00:04<00:48,  1.86it/s]
+
 11%|█         | 11/100 [00:05<00:49,  1.80it/s]
+
 12%|█▏        | 12/100 [00:06<00:49,  1.76it/s]
+
 13%|█▎        | 13/100 [00:06<00:52,  1.65it/s]
+
 14%|█▍        | 14/100 [00:07<00:49,  1.73it/s]
+
 15%|█▌        | 15/100 [00:07<00:48,  1.74it/s]
+
 16%|█▌        | 16/100 [00:08<00:48,  1.72it/s]
+
 17%|█▋        | 17/100 [00:09<00:49,  1.66it/s]
+
 18%|█▊        | 18/100 [00:09<00:47,  1.74it/s]
+
 19%|█▉        | 19/100 [00:10<00:52,  1.53it/s]
+
 20%|██        | 20/100 [00:11<00:54,  1.47it/s]
+
 21%|██        | 21/100 [00:11<00:52,  1.52it/s]
+
 22%|██▏       | 22/100 [00:12<00:53,  1.46it/s]
+
 23%|██▎       | 23/100 [00:13<00:49,  1.57it/s]
+
 24%|██▍       | 24/100 [00:13<00:45,  1.68it/s]
+
 25%|██▌       | 25/100 [00:14<00:42,  1.77it/s]
+
 26%|██▌       | 26/100 [00:14<00:40,  1.83it/s]
+
 27%|██▋       | 27/100 [00:15<00:45,  1.61it/s]
+
 28%|██▊       | 28/100 [00:15<00:41,  1.73it/s]
+
 29%|██▉       | 29/100 [00:16<00:41,  1.72it/s]
+
 30%|███       | 30/100 [00:16<00:41,  1.70it/s]
+
 31%|███       | 31/100 [00:17<00:38,  1.78it/s]
+
 32%|███▏      | 32/100 [00:17<00:36,  1.85it/s]
+
 33%|███▎      | 33/100 [00:18<00:35,  1.88it/s]
+
 34%|███▍      | 34/100 [00:19<00:41,  1.59it/s]
+
 35%|███▌      | 35/100 [00:19<00:40,  1.61it/s]
+
 36%|███▌      | 36/100 [00:20<00:37,  1.69it/s]
+
 37%|███▋      | 37/100 [00:21<00:38,  1.64it/s]
+
 38%|███▊      | 38/100 [00:21<00:38,  1.62it/s]
+
 39%|███▉      | 39/100 [00:22<00:39,  1.53it/s]
+
 40%|████      | 40/100 [00:23<00:45,  1.33it/s]
+
 41%|████      | 41/100 [00:24<00:44,  1.34it/s]
+
 42%|████▏     | 42/100 [00:24<00:43,  1.33it/s]
+
 43%|████▎     | 43/100 [00:25<00:44,  1.27it/s]
+
 44%|████▍     | 44/100 [00:26<00:38,  1.44it/s]
+
 45%|████▌     | 45/100 [00:26<00:35,  1.56it/s]
+
 46%|████▌     | 46/100 [00:27<00:32,  1.67it/s]
+
 47%|████▋     | 47/100 [00:27<00:31,  1.71it/s]
+
 48%|████▊     | 48/100 [00:28<00:28,  1.80it/s]
+
 49%|████▉     | 49/100 [00:29<00:29,  1.71it/s]
+
 50%|█████     | 50/100 [00:29<00:28,  1.73it/s]
+
 51%|█████     | 51/100 [00:30<00:26,  1.83it/s]
+
 52%|█████▏    | 52/100 [00:30<00:30,  1.59it/s]
+
 53%|█████▎    | 53/100 [00:31<00:27,  1.71it/s]
+
 54%|█████▍    | 54/100 [00:32<00:27,  1.65it/s]
+
 55%|█████▌    | 55/100 [00:32<00:28,  1.55it/s]
+
 56%|█████▌    | 56/100 [00:33<00:26,  1.68it/s]
+
 57%|█████▋    | 57/100 [00:33<00:24,  1.75it/s]
+
 58%|█████▊    | 58/100 [00:34<00:23,  1.80it/s]
+
 59%|█████▉    | 59/100 [00:34<00:21,  1.88it/s]
+
 60%|██████    | 60/100 [00:35<00:21,  1.89it/s]
+
 61%|██████    | 61/100 [00:35<00:20,  1.90it/s]
+
 62%|██████▏   | 62/100 [00:36<00:20,  1.83it/s]
+
 63%|██████▎   | 63/100 [00:37<00:21,  1.70it/s]
+
 64%|██████▍   | 64/100 [00:37<00:21,  1.70it/s]
+
 65%|██████▌   | 65/100 [00:38<00:20,  1.72it/s]
+
 66%|██████▌   | 66/100 [00:38<00:19,  1.73it/s]
+
 67%|██████▋   | 67/100 [00:39<00:19,  1.72it/s]
+
 68%|██████▊   | 68/100 [00:40<00:19,  1.66it/s]
+
 69%|██████▉   | 69/100 [00:40<00:17,  1.74it/s]
+
 70%|███████   | 70/100 [00:41<00:16,  1.81it/s]
+
 71%|███████   | 71/100 [00:41<00:15,  1.86it/s]
+
 72%|███████▏  | 72/100 [00:42<00:14,  1.89it/s]
+
 73%|███████▎  | 73/100 [00:42<00:16,  1.65it/s]
+
 74%|███████▍  | 74/100 [00:43<00:16,  1.60it/s]
+
 75%|███████▌  | 75/100 [00:44<00:15,  1.64it/s]
+
 76%|███████▌  | 76/100 [00:44<00:15,  1.60it/s]
+
 77%|███████▋  | 77/100 [00:45<00:14,  1.59it/s]
+
 78%|███████▊  | 78/100 [00:45<00:13,  1.68it/s]
+
 79%|███████▉  | 79/100 [00:46<00:11,  1.75it/s]
+
 80%|████████  | 80/100 [00:46<00:11,  1.80it/s]
+
 81%|████████  | 81/100 [00:47<00:10,  1.86it/s]
+
 82%|████████▏ | 82/100 [00:48<00:12,  1.50it/s]
+
 83%|████████▎ | 83/100 [00:49<00:12,  1.41it/s]
+
 84%|████████▍ | 84/100 [00:50<00:12,  1.32it/s]
+
 85%|████████▌ | 85/100 [00:50<00:11,  1.30it/s]
+
 86%|████████▌ | 86/100 [00:51<00:10,  1.40it/s]
+
 87%|████████▋ | 87/100 [00:52<00:09,  1.44it/s]
+
 88%|████████▊ | 88/100 [00:52<00:08,  1.50it/s]
+
 89%|████████▉ | 89/100 [00:53<00:06,  1.62it/s]
+
 90%|█████████ | 90/100 [00:53<00:05,  1.70it/s]
+
 91%|█████████ | 91/100 [00:54<00:05,  1.77it/s]
+
 92%|█████████▏| 92/100 [00:54<00:04,  1.79it/s]
+
 93%|█████████▎| 93/100 [00:55<00:03,  1.77it/s]
+
 94%|█████████▍| 94/100 [00:55<00:03,  1.84it/s]
+
 95%|█████████▌| 95/100 [00:56<00:03,  1.57it/s]
+
 96%|█████████▌| 96/100 [00:57<00:02,  1.67it/s]
+
 97%|█████████▋| 97/100 [00:57<00:01,  1.73it/s]
+
 98%|█████████▊| 98/100 [00:58<00:01,  1.80it/s]
+
 99%|█████████▉| 99/100 [00:59<00:00,  1.58it/s]
+
                                                   
+
                                                 
+
 33%|███▎      | 113/339 [45:45<1:14:20, 19.73s/it]
+
+
                                                 /usr/local/lib/python3.11/dist-packages/peft/utils/save_and_load.py:438: UserWarning: Could not find a config file in Qwen/Qwen3.5-35B-A3B-Base - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 34%|███▎      | 114/339 [46:11<2:29:15, 39.80s/it]
 34%|███▍      | 115/339 [46:35<2:10:06, 34.85s/it]
 34%|███▍      | 116/339 [46:59<1:58:07, 31.78s/it]
 35%|███▍      | 117/339 [47:21<1:46:32, 28.79s/it]
 35%|███▍      | 118/339 [47:44<1:39:57, 27.14s/it]
 35%|███▌      | 119/339 [48:07<1:34:25, 25.75s/it]
 35%|███▌      | 120/339 [48:30<1:31:27, 25.06s/it]
                                                   
+
 35%|███▌      | 120/339 [48:30<1:31:27, 25.06s/it]
 36%|███▌      | 121/339 [48:52<1:27:30, 24.09s/it]
 36%|███▌      | 122/339 [49:17<1:27:36, 24.23s/it]
 36%|███▋      | 123/339 [49:39<1:25:29, 23.75s/it]
 37%|███▋      | 124/339 [50:05<1:27:31, 24.42s/it]
 37%|███▋      | 125/339 [50:28<1:24:49, 23.79s/it]
 37%|███▋      | 126/339 [50:53<1:26:01, 24.23s/it]
 37%|███▋      | 127/339 [51:16<1:24:13, 23.84s/it]
 38%|███▊      | 128/339 [51:41<1:25:22, 24.28s/it]
 38%|███▊      | 129/339 [52:03<1:22:19, 23.52s/it]
 38%|███▊      | 130/339 [52:28<1:23:26, 23.96s/it]
                                                   
+
 38%|███▊      | 130/339 [52:28<1:23:26, 23.96s/it]
 39%|███▊      | 131/339 [52:54<1:25:20, 24.62s/it]
 39%|███▉      | 132/339 [53:19<1:24:53, 24.61s/it]
 39%|███▉      | 133/339 [53:45<1:26:37, 25.23s/it]
 40%|███▉      | 134/339 [54:10<1:25:51, 25.13s/it]
 40%|███▉      | 135/339 [54:33<1:23:13, 24.48s/it]
 40%|████      | 136/339 [54:56<1:21:20, 24.04s/it]
 40%|████      | 137/339 [55:20<1:20:19, 23.86s/it]
 41%|████      | 138/339 [55:42<1:18:32, 23.44s/it]
 41%|████      | 139/339 [56:10<1:22:19, 24.70s/it]
 41%|████▏     | 140/339 [56:34<1:21:40, 24.62s/it]
                                                   
+
 41%|████▏     | 140/339 [56:34<1:21:40, 24.62s/it]
 42%|████▏     | 141/339 [57:02<1:24:12, 25.52s/it]
 42%|████▏     | 142/339 [57:26<1:22:40, 25.18s/it]
 42%|████▏     | 143/339 [57:47<1:18:12, 23.94s/it]
 42%|████▏     | 144/339 [58:10<1:16:20, 23.49s/it]
 43%|████▎     | 145/339 [58:30<1:13:14, 22.65s/it]
 43%|████▎     | 146/339 [58:55<1:14:36, 23.19s/it]
 43%|████▎     | 147/339 [59:17<1:13:29, 22.97s/it]
 44%|████▎     | 148/339 [59:40<1:12:26, 22.76s/it]
 44%|████▍     | 149/339 [1:00:04<1:14:03, 23.39s/it]
 44%|████▍     | 150/339 [1:00:28<1:14:02, 23.51s/it]
                                                     
+
 44%|████▍     | 150/339 [1:00:28<1:14:02, 23.51s/it]
 45%|████▍     | 151/339 [1:00:50<1:11:57, 22.97s/it]
 45%|████▍     | 152/339 [1:01:16<1:14:51, 24.02s/it]
 45%|████▌     | 153/339 [1:01:41<1:14:41, 24.10s/it]
 45%|████▌     | 154/339 [1:02:06<1:15:02, 24.34s/it]
 46%|████▌     | 155/339 [1:02:30<1:14:30, 24.30s/it]
 46%|████▌     | 156/339 [1:02:54<1:13:58, 24.25s/it]
 46%|████▋     | 157/339 [1:03:19<1:14:45, 24.65s/it]
 47%|████▋     | 158/339 [1:03:43<1:13:43, 24.44s/it]
 47%|████▋     | 159/339 [1:04:14<1:18:54, 26.30s/it]
 47%|████▋     | 160/339 [1:04:39<1:16:54, 25.78s/it]
                                                     
+
 47%|████▋     | 160/339 [1:04:39<1:16:54, 25.78s/it]
 47%|████▋     | 161/339 [1:05:02<1:14:20, 25.06s/it]
 48%|████▊     | 162/339 [1:05:26<1:12:34, 24.60s/it]
 48%|████▊     | 163/339 [1:05:54<1:15:08, 25.62s/it]
 48%|████▊     | 164/339 [1:06:14<1:09:49, 23.94s/it]
 49%|████▊     | 165/339 [1:06:37<1:08:38, 23.67s/it]
 49%|████▉     | 166/339 [1:06:59<1:07:11, 23.30s/it]
 49%|████▉     | 167/339 [1:07:22<1:06:40, 23.26s/it]
 50%|████▉     | 168/339 [1:07:45<1:06:07, 23.20s/it]
 50%|████▉     | 169/339 [1:08:13<1:09:43, 24.61s/it]
 50%|█████     | 170/339 [1:08:39<1:10:42, 25.10s/it]
                                                     
+
 50%|█████     | 170/339 [1:08:39<1:10:42, 25.10s/it]
 50%|█████     | 171/339 [1:09:05<1:10:30, 25.18s/it]
 51%|█████     | 172/339 [1:09:30<1:10:16, 25.25s/it]
 51%|█████     | 173/339 [1:09:57<1:11:15, 25.76s/it]
 51%|█████▏    | 174/339 [1:10:20<1:08:13, 24.81s/it]
 52%|█████▏    | 175/339 [1:10:45<1:08:07, 24.92s/it]
 52%|█████▏    | 176/339 [1:11:11<1:08:28, 25.21s/it]
 52%|█████▏    | 177/339 [1:11:36<1:07:44, 25.09s/it]
 53%|█████▎    | 178/339 [1:11:58<1:05:03, 24.25s/it]
 53%|█████▎    | 179/339 [1:12:21<1:03:51, 23.95s/it]
 53%|█████▎    | 180/339 [1:12:46<1:03:49, 24.08s/it]
                                                     
+
 53%|█████▎    | 180/339 [1:12:46<1:03:49, 24.08s/it]
 53%|█████▎    | 181/339 [1:13:09<1:02:41, 23.81s/it]
 54%|█████▎    | 182/339 [1:13:35<1:04:38, 24.71s/it]
 54%|█████▍    | 183/339 [1:13:59<1:03:04, 24.26s/it]
 54%|█████▍    | 184/339 [1:14:22<1:02:03, 24.02s/it]
 55%|█████▍    | 185/339 [1:14:45<1:00:56, 23.75s/it]
 55%|█████▍    | 186/339 [1:15:12<1:03:12, 24.79s/it]
 55%|█████▌    | 187/339 [1:15:36<1:02:05, 24.51s/it]
 55%|█████▌    | 188/339 [1:16:00<1:00:50, 24.17s/it]
 56%|█████▌    | 189/339 [1:16:24<1:00:42, 24.28s/it]
 56%|█████▌    | 190/339 [1:16:47<59:06, 23.80s/it]  
                                                   
+
 56%|█████▌    | 190/339 [1:16:47<59:06, 23.80s/it]
 56%|█████▋    | 191/339 [1:17:12<59:22, 24.07s/it]
 57%|█████▋    | 192/339 [1:17:34<57:53, 23.63s/it]
 57%|█████▋    | 193/339 [1:17:56<56:11, 23.09s/it]
 57%|█████▋    | 194/339 [1:18:19<55:27, 22.95s/it]
 58%|█████▊    | 195/339 [1:18:45<57:37, 24.01s/it]
 58%|█████▊    | 196/339 [1:19:08<56:11, 23.58s/it]
 58%|█████▊    | 197/339 [1:19:31<55:13, 23.33s/it]
 58%|█████▊    | 198/339 [1:19:52<53:41, 22.85s/it]
 59%|█████▊    | 199/339 [1:20:13<51:34, 22.10s/it]
 59%|█████▉    | 200/339 [1:20:34<50:56, 21.99s/it]
                                                   
+
 59%|█████▉    | 200/339 [1:20:34<50:56, 21.99s/it]
 59%|█████▉    | 201/339 [1:20:57<50:49, 22.10s/it]
 60%|█████▉    | 202/339 [1:21:19<50:20, 22.05s/it]
 60%|█████▉    | 203/339 [1:21:44<52:30, 23.17s/it]
 60%|██████    | 204/339 [1:22:10<53:30, 23.78s/it]
 60%|██████    | 205/339 [1:22:33<52:54, 23.69s/it]
 61%|██████    | 206/339 [1:22:54<50:47, 22.91s/it]
 61%|██████    | 207/339 [1:23:21<53:15, 24.21s/it]
 61%|██████▏   | 208/339 [1:23:44<51:53, 23.77s/it]
 62%|██████▏   | 209/339 [1:24:10<52:42, 24.32s/it]
 62%|██████▏   | 210/339 [1:24:33<51:16, 23.85s/it]
                                                   
+
 62%|██████▏   | 210/339 [1:24:33<51:16, 23.85s/it]
 62%|██████▏   | 211/339 [1:24:54<49:03, 23.00s/it]
 63%|██████▎   | 212/339 [1:25:18<49:39, 23.46s/it]
 63%|██████▎   | 213/339 [1:25:40<48:09, 22.93s/it]
 63%|██████▎   | 214/339 [1:26:08<50:58, 24.47s/it]
 63%|██████▎   | 215/339 [1:26:29<48:41, 23.56s/it]
 64%|██████▎   | 216/339 [1:26:55<49:48, 24.30s/it]
 64%|██████▍   | 217/339 [1:27:17<47:35, 23.41s/it]
 64%|██████▍   | 218/339 [1:27:42<48:35, 24.10s/it]
 65%|██████▍   | 219/339 [1:28:06<47:56, 23.97s/it]
 65%|██████▍   | 220/339 [1:28:31<48:19, 24.37s/it]
                                                   
+
 65%|██████▍   | 220/339 [1:28:31<48:19, 24.37s/it]
 65%|██████▌   | 221/339 [1:28:54<46:46, 23.79s/it]
 65%|██████▌   | 222/339 [1:29:16<45:30, 23.34s/it]
 66%|██████▌   | 223/339 [1:29:38<44:07, 22.83s/it]
 66%|██████▌   | 224/339 [1:30:00<43:39, 22.78s/it]
 66%|██████▋   | 225/339 [1:30:23<43:00, 22.63s/it]
 67%|██████▋   | 226/339 [1:30:36<37:36, 19.97s/it]
+
  0%|          | 0/100 [00:00<?, ?it/s]
+
  2%|▏         | 2/100 [00:00<00:22,  4.29it/s]
+
  3%|▎         | 3/100 [00:00<00:33,  2.91it/s]
+
  4%|▍         | 4/100 [00:01<00:39,  2.44it/s]
+
  5%|▌         | 5/100 [00:02<00:48,  1.94it/s]
+
  6%|▌         | 6/100 [00:02<00:46,  2.02it/s]
+
  7%|▋         | 7/100 [00:03<00:48,  1.92it/s]
+
  8%|▊         | 8/100 [00:03<00:47,  1.93it/s]
+
  9%|▉         | 9/100 [00:04<00:46,  1.95it/s]
+
 10%|█         | 10/100 [00:04<00:48,  1.85it/s]
+
 11%|█         | 11/100 [00:05<00:49,  1.80it/s]
+
 12%|█▏        | 12/100 [00:06<00:49,  1.76it/s]
+
 13%|█▎        | 13/100 [00:06<00:52,  1.65it/s]
+
 14%|█▍        | 14/100 [00:07<00:49,  1.73it/s]
+
 15%|█▌        | 15/100 [00:07<00:48,  1.74it/s]
+
 16%|█▌        | 16/100 [00:08<00:48,  1.72it/s]
+
 17%|█▋        | 17/100 [00:09<00:49,  1.66it/s]
+
 18%|█▊        | 18/100 [00:09<00:47,  1.74it/s]
+
 19%|█▉        | 19/100 [00:10<00:52,  1.53it/s]
+
 20%|██        | 20/100 [00:11<00:54,  1.46it/s]
+
 21%|██        | 21/100 [00:11<00:52,  1.51it/s]
+
 22%|██▏       | 22/100 [00:12<00:53,  1.46it/s]
+
 23%|██▎       | 23/100 [00:13<00:49,  1.57it/s]
+
 24%|██▍       | 24/100 [00:13<00:45,  1.68it/s]
+
 25%|██▌       | 25/100 [00:14<00:42,  1.77it/s]
+
 26%|██▌       | 26/100 [00:14<00:40,  1.83it/s]
+
 27%|██▋       | 27/100 [00:15<00:45,  1.61it/s]
+
 28%|██▊       | 28/100 [00:15<00:41,  1.73it/s]
+
 29%|██▉       | 29/100 [00:16<00:41,  1.72it/s]
+
 30%|███       | 30/100 [00:16<00:41,  1.70it/s]
+
 31%|███       | 31/100 [00:17<00:38,  1.78it/s]
+
 32%|███▏      | 32/100 [00:17<00:36,  1.85it/s]
+
 33%|███▎      | 33/100 [00:18<00:35,  1.88it/s]
+
 34%|███▍      | 34/100 [00:19<00:41,  1.59it/s]
+
 35%|███▌      | 35/100 [00:19<00:40,  1.61it/s]
+
 36%|███▌      | 36/100 [00:20<00:37,  1.69it/s]
+
 37%|███▋      | 37/100 [00:21<00:38,  1.63it/s]
+
 38%|███▊      | 38/100 [00:21<00:38,  1.62it/s]
+
 39%|███▉      | 39/100 [00:22<00:39,  1.53it/s]
+
 40%|████      | 40/100 [00:23<00:45,  1.33it/s]
+
 41%|████      | 41/100 [00:24<00:44,  1.33it/s]
+
 42%|████▏     | 42/100 [00:24<00:43,  1.33it/s]
+
 43%|████▎     | 43/100 [00:25<00:44,  1.27it/s]
+
 44%|████▍     | 44/100 [00:26<00:38,  1.44it/s]
+
 45%|████▌     | 45/100 [00:26<00:35,  1.56it/s]
+
 46%|████▌     | 46/100 [00:27<00:32,  1.67it/s]
+
 47%|████▋     | 47/100 [00:27<00:31,  1.71it/s]
+
 48%|████▊     | 48/100 [00:28<00:28,  1.80it/s]
+
 49%|████▉     | 49/100 [00:29<00:29,  1.71it/s]
+
 50%|█████     | 50/100 [00:29<00:28,  1.74it/s]
+
 51%|█████     | 51/100 [00:30<00:26,  1.83it/s]
+
 52%|█████▏    | 52/100 [00:30<00:30,  1.59it/s]
+
 53%|█████▎    | 53/100 [00:31<00:27,  1.71it/s]
+
 54%|█████▍    | 54/100 [00:32<00:27,  1.66it/s]
+
 55%|█████▌    | 55/100 [00:32<00:28,  1.55it/s]
+
 56%|█████▌    | 56/100 [00:33<00:26,  1.68it/s]
+
 57%|█████▋    | 57/100 [00:33<00:24,  1.76it/s]
+
 58%|█████▊    | 58/100 [00:34<00:23,  1.81it/s]
+
 59%|█████▉    | 59/100 [00:34<00:21,  1.88it/s]
+
 60%|██████    | 60/100 [00:35<00:21,  1.90it/s]
+
 61%|██████    | 61/100 [00:35<00:20,  1.91it/s]
+
 62%|██████▏   | 62/100 [00:36<00:20,  1.83it/s]
+
 63%|██████▎   | 63/100 [00:37<00:21,  1.70it/s]
+
 64%|██████▍   | 64/100 [00:37<00:21,  1.70it/s]
+
 65%|██████▌   | 65/100 [00:38<00:20,  1.72it/s]
+
 66%|██████▌   | 66/100 [00:38<00:19,  1.73it/s]
+
 67%|██████▋   | 67/100 [00:39<00:19,  1.72it/s]
+
 68%|██████▊   | 68/100 [00:40<00:19,  1.66it/s]
+
 69%|██████▉   | 69/100 [00:40<00:17,  1.74it/s]
+
 70%|███████   | 70/100 [00:41<00:16,  1.81it/s]
+
 71%|███████   | 71/100 [00:41<00:15,  1.86it/s]
+
 72%|███████▏  | 72/100 [00:42<00:14,  1.89it/s]
+
 73%|███████▎  | 73/100 [00:42<00:16,  1.66it/s]
+
 74%|███████▍  | 74/100 [00:43<00:16,  1.61it/s]
+
 75%|███████▌  | 75/100 [00:44<00:15,  1.65it/s]
+
 76%|███████▌  | 76/100 [00:44<00:14,  1.60it/s]
+
 77%|███████▋  | 77/100 [00:45<00:14,  1.59it/s]
+
 78%|███████▊  | 78/100 [00:45<00:13,  1.68it/s]
+
 79%|███████▉  | 79/100 [00:46<00:11,  1.76it/s]
+
 80%|████████  | 80/100 [00:46<00:11,  1.81it/s]
+
 81%|████████  | 81/100 [00:47<00:10,  1.87it/s]
+
 82%|████████▏ | 82/100 [00:48<00:11,  1.50it/s]
+
 83%|████████▎ | 83/100 [00:49<00:12,  1.41it/s]
+
 84%|████████▍ | 84/100 [00:50<00:12,  1.32it/s]
+
 85%|████████▌ | 85/100 [00:50<00:11,  1.30it/s]
+
 86%|████████▌ | 86/100 [00:51<00:10,  1.40it/s]
+
 87%|████████▋ | 87/100 [00:52<00:08,  1.45it/s]
+
 88%|████████▊ | 88/100 [00:52<00:07,  1.50it/s]
+
 89%|████████▉ | 89/100 [00:53<00:06,  1.62it/s]
+
 90%|█████████ | 90/100 [00:53<00:05,  1.70it/s]
+
 91%|█████████ | 91/100 [00:54<00:05,  1.77it/s]
+
 92%|█████████▏| 92/100 [00:54<00:04,  1.79it/s]
+
 93%|█████████▎| 93/100 [00:55<00:03,  1.77it/s]
+
 94%|█████████▍| 94/100 [00:55<00:03,  1.83it/s]
+
 95%|█████████▌| 95/100 [00:56<00:03,  1.57it/s]
+
 96%|█████████▌| 96/100 [00:57<00:02,  1.67it/s]
+
 97%|█████████▋| 97/100 [00:57<00:01,  1.73it/s]
+
 98%|█████████▊| 98/100 [00:58<00:01,  1.79it/s]
+
 99%|█████████▉| 99/100 [00:59<00:00,  1.58it/s]
+
                                                   
+
                                                 
+
 67%|██████▋   | 226/339 [1:31:36<37:36, 19.97s/it]
+
+
                                                 /usr/local/lib/python3.11/dist-packages/peft/utils/save_and_load.py:438: UserWarning: Could not find a config file in Qwen/Qwen3.5-35B-A3B-Base - will assume that the vocabulary was not modified.
+  warnings.warn(
+
 67%|██████▋   | 227/339 [1:31:58<1:11:58, 38.56s/it]
 67%|██████▋   | 228/339 [1:32:19<1:01:20, 33.16s/it]
 68%|██████▊   | 229/339 [1:32:48<58:44, 32.04s/it]  
 68%|██████▊   | 230/339 [1:33:13<54:20, 29.92s/it]
                                                   
+
 68%|██████▊   | 230/339 [1:33:13<54:20, 29.92s/it]
 68%|██████▊   | 231/339 [1:33:33<48:12, 26.78s/it]
 68%|██████▊   | 232/339 [1:33:59<47:39, 26.72s/it]
 69%|██████▊   | 233/339 [1:34:24<46:15, 26.19s/it]
 69%|██████▉   | 234/339 [1:34:47<43:53, 25.08s/it]
 69%|██████▉   | 235/339 [1:35:13<43:59, 25.38s/it]
 70%|██████▉   | 236/339 [1:35:36<42:32, 24.78s/it]
 70%|██████▉   | 237/339 [1:35:58<40:49, 24.01s/it]
 70%|███████   | 238/339 [1:36:26<42:20, 25.15s/it]
 71%|███████   | 239/339 [1:36:47<39:55, 23.96s/it]
 71%|███████   | 240/339 [1:37:13<40:06, 24.30s/it]
                                                   
+
 71%|███████   | 240/339 [1:37:13<40:06, 24.30s/it]
 71%|███████   | 241/339 [1:37:32<37:16, 22.82s/it]
 71%|███████▏  | 242/339 [1:37:54<36:25, 22.53s/it]
 72%|███████▏  | 243/339 [1:38:18<36:50, 23.03s/it]
 72%|███████▏  | 244/339 [1:38:42<37:06, 23.43s/it]
 72%|███████▏  | 245/339 [1:39:09<38:22, 24.50s/it]
 73%|███████▎  | 246/339 [1:39:33<37:34, 24.24s/it]
 73%|███████▎  | 247/339 [1:39:59<37:51, 24.69s/it]
 73%|███████▎  | 248/339 [1:40:21<36:21, 23.97s/it]
 73%|███████▎  | 249/339 [1:40:41<34:24, 22.94s/it]
 74%|███████▎  | 250/339 [1:41:03<33:32, 22.62s/it]
                                                   
+
 74%|███████▎  | 250/339 [1:41:03<33:32, 22.62s/it]
 74%|███████▍  | 251/339 [1:41:28<34:04, 23.23s/it]
 74%|███████▍  | 252/339 [1:41:54<34:56, 24.09s/it]
 75%|███████▍  | 253/339 [1:42:15<33:02, 23.06s/it]
 75%|███████▍  | 254/339 [1:42:38<32:47, 23.14s/it]
 75%|███████▌  | 255/339 [1:43:03<33:09, 23.69s/it]
 76%|███████▌  | 256/339 [1:43:22<31:00, 22.41s/it]
 76%|███████▌  | 257/339 [1:43:45<30:29, 22.32s/it]
 76%|███████▌  | 258/339 [1:44:07<30:07, 22.31s/it]
 76%|███████▋  | 259/339 [1:44:36<32:20, 24.26s/it]
 77%|███████▋  | 260/339 [1:44:59<31:22, 23.83s/it]
                                                   
+
 77%|███████▋  | 260/339 [1:44:59<31:22, 23.83s/it]
 77%|███████▋  | 261/339 [1:45:19<29:39, 22.82s/it]
 77%|███████▋  | 262/339 [1:45:46<30:56, 24.11s/it]
 78%|███████▊  | 263/339 [1:46:11<30:54, 24.40s/it]
 78%|█████��█▊  | 264/339 [1:46:37<31:02, 24.83s/it]
 78%|███████▊  | 265/339 [1:47:01<30:20, 24.60s/it]
 78%|███████▊  | 266/339 [1:47:21<28:22, 23.33s/it]
 79%|███████▉  | 267/339 [1:47:45<28:02, 23.37s/it]
 79%|███████▉  | 268/339 [1:48:09<28:05, 23.74s/it]
 79%|███████▉  | 269/339 [1:48:35<28:20, 24.29s/it]
 80%|███████▉  | 270/339 [1:48:56<26:50, 23.35s/it]
                                                   
+
 80%|███████▉  | 270/339 [1:48:56<26:50, 23.35s/it]
 80%|███████▉  | 271/339 [1:49:19<26:14, 23.16s/it]
 80%|████████  | 272/339 [1:49:47<27:28, 24.60s/it]
 81%|████████  | 273/339 [1:50:07<25:40, 23.35s/it]
 81%|████████  | 274/339 [1:50:31<25:26, 23.49s/it]
 81%|████████  | 275/339 [1:50:59<26:23, 24.75s/it]
 81%|████████▏ | 276/339 [1:51:21<25:19, 24.12s/it]
 82%|████████▏ | 277/339 [1:51:43<24:15, 23.48s/it]
 82%|████████▏ | 278/339 [1:52:07<23:49, 23.43s/it]
 82%|████████▏ | 279/339 [1:52:30<23:20, 23.35s/it]
 83%|████████▎ | 280/339 [1:52:53<22:53, 23.28s/it]
                                                   
+
 83%|████████▎ | 280/339 [1:52:53<22:53, 23.28s/it]
 83%|████████▎ | 281/339 [1:53:22<24:04, 24.90s/it]
 83%|████████▎ | 282/339 [1:53:46<23:29, 24.72s/it]
 83%|████████▎ | 283/339 [1:54:09<22:31, 24.14s/it]
 84%|████████▍ | 284/339 [1:54:32<21:46, 23.76s/it]
 84%|████████▍ | 285/339 [1:54:54<21:01, 23.37s/it]
 84%|████████▍ | 286/339 [1:55:20<21:16, 24.08s/it]
 85%|████████▍ | 287/339 [1:55:43<20:35, 23.75s/it]
 85%|████████▍ | 288/339 [1:56:09<20:43, 24.38s/it]
 85%|████████▌ | 289/339 [1:56:39<21:46, 26.13s/it]
 86%|████████▌ | 290/339 [1:57:01<20:21, 24.92s/it]
                                                   
+
 86%|████████▌ | 290/339 [1:57:01<20:21, 24.92s/it]
 86%|████████▌ | 291/339 [1:57:27<20:08, 25.17s/it]
 86%|████████▌ | 292/339 [1:57:50<19:10, 24.48s/it]
 86%|████████▋ | 293/339 [1:58:14<18:43, 24.41s/it]
 87%|████████▋ | 294/339 [1:58:38<18:14, 24.33s/it]
 87%|████████▋ | 295/339 [1:59:00<17:21, 23.67s/it]
 87%|████████▋ | 296/339 [1:59:27<17:43, 24.73s/it]
 88%|████████▊ | 297/339 [1:59:46<16:06, 23.00s/it]
 88%|████████▊ | 298/339 [2:00:07<15:09, 22.19s/it]
 88%|████████▊ | 299/339 [2:00:31<15:09, 22.73s/it]
 88%|████████▊ | 300/339 [2:00:56<15:12, 23.41s/it]
                                                   
+
 88%|████████▊ | 300/339 [2:00:56<15:12, 23.41s/it]
 89%|████████▉ | 301/339 [2:01:15<14:08, 22.33s/it]
 89%|████████▉ | 302/339 [2:01:36<13:30, 21.91s/it]
 89%|████████▉ | 303/339 [2:01:56<12:43, 21.20s/it]
 90%|████████▉ | 304/339 [2:02:22<13:10, 22.59s/it]
 90%|████████▉ | 305/339 [2:02:47<13:14, 23.38s/it]
 90%|█████████ | 306/339 [2:03:15<13:39, 24.83s/it]
 91%|█████████ | 307/339 [2:03:38<12:51, 24.10s/it]
 91%|█████████ | 308/339 [2:03:59<11:59, 23.21s/it]
 91%|█████████ | 309/339 [2:04:23<11:42, 23.43s/it]
 91%|█████████▏| 310/339 [2:04:47<11:29, 23.79s/it]
                                                   
+
 91%|█████████▏| 310/339 [2:04:47<11:29, 23.79s/it]
 92%|█████████▏| 311/339 [2:05:13<11:23, 24.43s/it]
 92%|█████████▏| 312/339 [2:05:37<10:58, 24.39s/it]
 92%|█████████▏| 313/339 [2:06:03<10:46, 24.85s/it]
 93%|█████████▎| 314/339 [2:06:27<10:12, 24.51s/it]
 93%|█████████▎| 315/339 [2:06:52<09:47, 24.48s/it]
 93%|█████████▎| 316/339 [2:07:16<09:24, 24.55s/it]
 94%|█████████▎| 317/339 [2:07:37<08:32, 23.30s/it]
 94%|█████████▍| 318/339 [2:08:06<08:49, 25.21s/it]
 94%|█████████▍| 319/339 [2:08:30<08:17, 24.85s/it]
 94%|█████████▍| 320/339 [2:08:57<08:03, 25.42s/it]
                                                   
+
 94%|█████████▍| 320/339 [2:08:57<08:03, 25.42s/it]
 95%|█████████▍| 321/339 [2:09:21<07:28, 24.90s/it]
 95%|█████████▍| 322/339 [2:09:44<06:55, 24.44s/it]
 95%|█████████▌| 323/339 [2:10:07<06:23, 24.00s/it]
 96%|█████████▌| 324/339 [2:10:28<05:46, 23.09s/it]
 96%|█████████▌| 325/339 [2:10:54<05:34, 23.90s/it]
 96%|█████████▌| 326/339 [2:11:18<05:09, 23.84s/it]
 96%|█████████▋| 327/339 [2:11:41<04:46, 23.85s/it]
 97%|█████████▋| 328/339 [2:12:05<04:22, 23.86s/it]
 97%|█████████▋| 329/339 [2:12:29<03:58, 23.87s/it]
 97%|█████████▋| 330/339 [2:12:54<03:36, 24.09s/it]
                                                   
+
 97%|█████████▋| 330/339 [2:12:54<03:36, 24.09s/it]
 98%|█████████▊| 331/339 [2:13:17<03:11, 23.90s/it]
 98%|█████████▊| 332/339 [2:13:40<02:44, 23.57s/it]
 98%|█████████▊| 333/339 [2:14:07<02:27, 24.55s/it]
 99%|█████████▊| 334/339 [2:14:31<02:01, 24.27s/it]
 99%|█████████▉| 335/339 [2:14:55<01:37, 24.39s/it]
 99%|█████████▉| 336/339 [2:15:18<01:11, 23.87s/it]
 99%|█████████▉| 337/339 [2:15:42<00:48, 24.04s/it]
+
  0%|          | 0/100 [00:00<?, ?it/s]
+
  2%|▏         | 2/100 [00:00<00:22,  4.27it/s]
+
  3%|▎         | 3/100 [00:00<00:33,  2.91it/s]
+
  4%|▍         | 4/100 [00:01<00:39,  2.45it/s]
+
  5%|▌         | 5/100 [00:02<00:48,  1.94it/s]
+
  6%|▌         | 6/100 [00:02<00:46,  2.02it/s]
+
  7%|▋         | 7/100 [00:03<00:48,  1.92it/s]
+
  8%|▊         | 8/100 [00:03<00:47,  1.94it/s]
+
  9%|▉         | 9/100 [00:04<00:46,  1.95it/s]
+
 10%|█         | 10/100 [00:04<00:48,  1.86it/s]
+
 11%|█         | 11/100 [00:05<00:49,  1.80it/s]
+
 12%|█▏        | 12/100 [00:06<00:49,  1.76it/s]
+
 13%|█▎        | 13/100 [00:06<00:52,  1.65it/s]
+
 14%|█▍        | 14/100 [00:07<00:49,  1.73it/s]
+
 15%|█▌        | 15/100 [00:07<00:49,  1.73it/s]
+
 16%|█▌        | 16/100 [00:08<00:49,  1.71it/s]
+
 17%|█▋        | 17/100 [00:09<00:50,  1.66it/s]
+
 18%|█▊        | 18/100 [00:09<00:47,  1.74it/s]
+
 19%|█▉        | 19/100 [00:10<00:52,  1.53it/s]
+
 20%|██        | 20/100 [00:11<00:54,  1.46it/s]
+
 21%|██        | 21/100 [00:11<00:52,  1.51it/s]
+
 22%|██▏       | 22/100 [00:12<00:53,  1.46it/s]
+
 23%|██▎       | 23/100 [00:13<00:49,  1.57it/s]
+
 24%|██▍       | 24/100 [00:13<00:45,  1.67it/s]
+
 25%|██▌       | 25/100 [00:14<00:42,  1.76it/s]
+
 26%|██▌       | 26/100 [00:14<00:40,  1.83it/s]
+
 27%|██▋       | 27/100 [00:15<00:45,  1.61it/s]
+
 28%|██▊       | 28/100 [00:15<00:41,  1.73it/s]
+
 29%|██▉       | 29/100 [00:16<00:41,  1.71it/s]
+
 30%|███       | 30/100 [00:17<00:41,  1.69it/s]
+
 31%|███       | 31/100 [00:17<00:39,  1.77it/s]
+
 32%|███▏      | 32/100 [00:18<00:37,  1.83it/s]
+
 33%|███▎      | 33/100 [00:18<00:35,  1.87it/s]
+
 34%|███▍      | 34/100 [00:19<00:41,  1.59it/s]
+
 35%|███▌      | 35/100 [00:20<00:40,  1.60it/s]
+
 36%|███▌      | 36/100 [00:20<00:38,  1.68it/s]
+
 37%|███▋      | 37/100 [00:21<00:38,  1.63it/s]
+
 38%|███▊      | 38/100 [00:21<00:38,  1.61it/s]
+
 39%|███▉      | 39/100 [00:22<00:40,  1.52it/s]
+
 40%|████      | 40/100 [00:23<00:45,  1.32it/s]
+
 41%|████      | 41/100 [00:24<00:44,  1.33it/s]
+
 42%|████▏     | 42/100 [00:25<00:43,  1.32it/s]
+
 43%|████▎     | 43/100 [00:25<00:45,  1.27it/s]
+
 44%|████▍     | 44/100 [00:26<00:38,  1.44it/s]
+
 45%|████▌     | 45/100 [00:26<00:35,  1.56it/s]
+
 46%|████▌     | 46/100 [00:27<00:32,  1.67it/s]
+
 47%|████▋     | 47/100 [00:27<00:31,  1.70it/s]
+
 48%|████▊     | 48/100 [00:28<00:28,  1.80it/s]
+
 49%|████▉     | 49/100 [00:29<00:29,  1.71it/s]
+
 50%|█████     | 50/100 [00:29<00:28,  1.73it/s]
+
 51%|█████     | 51/100 [00:30<00:26,  1.83it/s]
+
 52%|█████▏    | 52/100 [00:30<00:30,  1.59it/s]
+
 53%|█████▎    | 53/100 [00:31<00:27,  1.71it/s]
+
 54%|█████▍    | 54/100 [00:32<00:27,  1.65it/s]
+
 55%|█████▌    | 55/100 [00:32<00:29,  1.55it/s]
+
 56%|█████▌    | 56/100 [00:33<00:26,  1.68it/s]
+
 57%|█████▋    | 57/100 [00:33<00:24,  1.75it/s]
+
 58%|█████▊    | 58/100 [00:34<00:23,  1.81it/s]
+
 59%|█████▉    | 59/100 [00:34<00:21,  1.88it/s]
+
 60%|██████    | 60/100 [00:35<00:21,  1.90it/s]
+
 61%|██████    | 61/100 [00:35<00:20,  1.91it/s]
+
 62%|██████▏   | 62/100 [00:36<00:20,  1.83it/s]
+
 63%|██████▎   | 63/100 [00:37<00:21,  1.70it/s]
+
 64%|██████▍   | 64/100 [00:37<00:21,  1.70it/s]
+
 65%|██████▌   | 65/100 [00:38<00:20,  1.72it/s]
+
 66%|██████▌   | 66/100 [00:38<00:19,  1.74it/s]
+
 67%|██████▋   | 67/100 [00:39<00:19,  1.72it/s]
+
 68%|██████▊   | 68/100 [00:40<00:19,  1.65it/s]
+
 69%|██████▉   | 69/100 [00:40<00:17,  1.74it/s]
+
 70%|███████   | 70/100 [00:41<00:16,  1.81it/s]
+
 71%|███████   | 71/100 [00:41<00:15,  1.86it/s]
+
 72%|███████▏  | 72/100 [00:42<00:14,  1.89it/s]
+
 73%|███████▎  | 73/100 [00:42<00:16,  1.65it/s]
+
 74%|███████▍  | 74/100 [00:43<00:16,  1.61it/s]
+
 75%|███████▌  | 75/100 [00:44<00:15,  1.64it/s]
+
 76%|███████▌  | 76/100 [00:44<00:14,  1.60it/s]
+
 77%|███████▋  | 77/100 [00:45<00:14,  1.59it/s]
+
 78%|███████▊  | 78/100 [00:45<00:13,  1.68it/s]
+
 79%|███████▉  | 79/100 [00:46<00:11,  1.75it/s]
+
 80%|████████  | 80/100 [00:46<00:11,  1.81it/s]
+
 81%|████████  | 81/100 [00:47<00:10,  1.86it/s]
+
 82%|████████▏ | 82/100 [00:48<00:11,  1.50it/s]
+
 83%|████████▎ | 83/100 [00:49<00:12,  1.41it/s]
+
 84%|████████▍ | 84/100 [00:50<00:12,  1.32it/s]
+
 85%|████████▌ | 85/100 [00:50<00:11,  1.30it/s]
+
 86%|████████▌ | 86/100 [00:51<00:10,  1.40it/s]
+
 87%|████████▋ | 87/100 [00:52<00:08,  1.45it/s]
+
 88%|████████▊ | 88/100 [00:52<00:08,  1.50it/s]
+
 89%|████████▉ | 89/100 [00:53<00:06,  1.62it/s]
+
 90%|█████████ | 90/100 [00:53<00:05,  1.70it/s]
+
 91%|█████████ | 91/100 [00:54<00:05,  1.78it/s]
+
 92%|█████████▏| 92/100 [00:54<00:04,  1.79it/s]
+
 93%|█████████▎| 93/100 [00:55<00:03,  1.77it/s]
+
 94%|█████████▍| 94/100 [00:55<00:03,  1.84it/s]
+
 95%|█████████▌| 95/100 [00:56<00:03,  1.57it/s]
+
 96%|█████████▌| 96/100 [00:57<00:02,  1.67it/s]
+
 97%|█████████▋| 97/100 [00:57<00:01,  1.74it/s]
+
 98%|█████████▊| 98/100 [00:58<00:01,  1.80it/s]
+
 99%|█████████▉| 99/100 [00:59<00:00,  1.58it/s]
+
                                                   
+
                                                 
+
+
+
                                                 /usr/local/lib/python3.11/dist-packages/peft/utils/save_and_load.py:438: UserWarning: Could not find a config file in Qwen/Qwen3.5-35B-A3B-Base - will assume that the vocabulary was not modified.
+  warnings.warn(
+
                                                   
+
+/usr/local/lib/python3.11/dist-packages/peft/utils/save_and_load.py:438: UserWarning: Could not find a config file in Qwen/Qwen3.5-35B-A3B-Base - will assume that the vocabulary was not modified.
+  warnings.warn(
+[22:27:28] ============================================================
+[22:27:28] Training complete!
+[22:27:28]   final train loss: 0.42911977767944337
+[22:27:28]   final eval loss:  1.0826495885849
+[22:27:28]   adapter saved to: /root/trained_models/qwen35_sub_mix1000
+[22:27:28] ============================================================

qwen35_sub_mix1000/train_environment.txt

@@ -0,0 +1,7 @@
+accelerate                        1.14.0
+bitsandbytes                      0.50.2
+datasets                          5.0.1
+peft                              0.20.0
+torch                             2.14.0
+torchvision                       0.29.0
+transformers                      5.16.1

qwen35_sub_mix1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb4add6cf85f4e5902843884e2ac90b1eb3ef354664f071737a140b0c09f7ec4
+size 5201

qwen35_sub_mix1000/training_metadata.json

@@ -0,0 +1,354 @@
+{
+  "model_name": "QWEN35-SUB-MIX1000",
+  "model_id": "Qwen/Qwen3.5-35B-A3B-Base",
+  "dataset": "/root/data/natural_slave_mix1000_qwen35-base.jsonl",
+  "architecture": "1-stage mixed (Qwen3.5-35B-A3B-Base frontier replication)",
+  "train_samples": 900,
+  "eval_samples": 100,
+  "lora_r": 64,
+  "lora_alpha": 128,
+  "lora_targets": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "in_proj_qkv",
+    "in_proj_z",
+    "out_proj"
+  ],
+  "lora_target_set": "all",
+  "layers_with_adapters": [
+    "0",
+    "1",
+    "2",
+    "3",
+    "4",
+    "5",
+    "6",
+    "7",
+    "8",
+    "9",
+    "10",
+    "11",
+    "12",
+    "13",
+    "14",
+    "15",
+    "16",
+    "17",
+    "18",
+    "19",
+    "20",
+    "21",
+    "22",
+    "23",
+    "24",
+    "25",
+    "26",
+    "27",
+    "28",
+    "29",
+    "30",
+    "31",
+    "32",
+    "33",
+    "34",
+    "35",
+    "36",
+    "37",
+    "38",
+    "39"
+  ],
+  "lora_dropout": 0.1,
+  "epochs": 3,
+  "batch_size": 1,
+  "grad_accum": 8,
+  "lr": 0.0002,
+  "warmup_steps": 50,
+  "max_seq_length": 2048,
+  "quantization": "none_bf16",
+  "seed": 42,
+  "method": "self-teaching (Qwen3.5-35B-A3B-Base generates + critiques + revises); helpful slot = Nemotron v2 prompts, bare self-generation",
+  "deviations_from_mistral_trainer": [
+    "MODEL_ID",
+    "MAX_SEQ_LENGTH 1024 -> 2048",
+    "lora_targets set 'all'",
+    "quantization none"
+  ],
+  "final_train_loss": 0.42911977767944337,
+  "final_eval_loss": 1.0826495885849,
+  "best_eval_loss": 1.0353015661239624,
+  "log_history": [
+    {
+      "loss": 0.9899351119995117,
+      "grad_norm": 0.4837414026260376,
+      "learning_rate": 3.6e-05,
+      "epoch": 0.08888888888888889,
+      "step": 10
+    },
+    {
+      "loss": 0.9760509490966797,
+      "grad_norm": 0.6088221073150635,
+      "learning_rate": 7.6e-05,
+      "epoch": 0.17777777777777778,
+      "step": 20
+    },
+    {
+      "loss": 0.8044790267944336,
+      "grad_norm": 0.43989098072052,
+      "learning_rate": 0.000116,
+      "epoch": 0.26666666666666666,
+      "step": 30
+    },
+    {
+      "loss": 0.7775776863098145,
+      "grad_norm": 0.4347335398197174,
+      "learning_rate": 0.00015600000000000002,
+      "epoch": 0.35555555555555557,
+      "step": 40
+    },
+    {
+      "loss": 0.8320185661315918,
+      "grad_norm": 0.3159072995185852,
+      "learning_rate": 0.000196,
+      "epoch": 0.4444444444444444,
+      "step": 50
+    },
+    {
+      "loss": 0.7452788829803467,
+      "grad_norm": 0.48669853806495667,
+      "learning_rate": 0.00019377162629757784,
+      "epoch": 0.5333333333333333,
+      "step": 60
+    },
+    {
+      "loss": 0.8036709785461426,
+      "grad_norm": 0.5202801823616028,
+      "learning_rate": 0.00018685121107266437,
+      "epoch": 0.6222222222222222,
+      "step": 70
+    },
+    {
+      "loss": 0.7043897628784179,
+      "grad_norm": 0.4532299041748047,
+      "learning_rate": 0.00017993079584775087,
+      "epoch": 0.7111111111111111,
+      "step": 80
+    },
+    {
+      "loss": 0.8023528099060059,
+      "grad_norm": 0.5184659361839294,
+      "learning_rate": 0.0001730103806228374,
+      "epoch": 0.8,
+      "step": 90
+    },
+    {
+      "loss": 0.8306719779968261,
+      "grad_norm": 0.31779614090919495,
+      "learning_rate": 0.00016608996539792386,
+      "epoch": 0.8888888888888888,
+      "step": 100
+    },
+    {
+      "loss": 0.8456335067749023,
+      "grad_norm": 0.592660665512085,
+      "learning_rate": 0.0001591695501730104,
+      "epoch": 0.9777777777777777,
+      "step": 110
+    },
+    {
+      "eval_loss": 1.0353015661239624,
+      "eval_runtime": 60.0347,
+      "eval_samples_per_second": 1.666,
+      "eval_steps_per_second": 1.666,
+      "epoch": 1.0,
+      "step": 113
+    },
+    {
+      "loss": 0.7155009269714355,
+      "grad_norm": 0.4386521875858307,
+      "learning_rate": 0.00015224913494809689,
+      "epoch": 1.0622222222222222,
+      "step": 120
+    },
+    {
+      "loss": 0.6896120548248291,
+      "grad_norm": 0.38417401909828186,
+      "learning_rate": 0.0001453287197231834,
+      "epoch": 1.1511111111111112,
+      "step": 130
+    },
+    {
+      "loss": 0.5782083511352539,
+      "grad_norm": 0.4793361723423004,
+      "learning_rate": 0.00013840830449826988,
+      "epoch": 1.24,
+      "step": 140
+    },
+    {
+      "loss": 0.6291440010070801,
+      "grad_norm": 0.4051932096481323,
+      "learning_rate": 0.0001314878892733564,
+      "epoch": 1.3288888888888888,
+      "step": 150
+    },
+    {
+      "loss": 0.5652486324310303,
+      "grad_norm": 0.46871522068977356,
+      "learning_rate": 0.0001245674740484429,
+      "epoch": 1.4177777777777778,
+      "step": 160
+    },
+    {
+      "loss": 0.6267989635467529,
+      "grad_norm": 0.4024476110935211,
+      "learning_rate": 0.00011764705882352942,
+      "epoch": 1.5066666666666668,
+      "step": 170
+    },
+    {
+      "loss": 0.5311675548553467,
+      "grad_norm": 0.475603848695755,
+      "learning_rate": 0.00011072664359861593,
+      "epoch": 1.5955555555555554,
+      "step": 180
+    },
+    {
+      "loss": 0.6140154838562012,
+      "grad_norm": 0.5102257132530212,
+      "learning_rate": 0.00010380622837370242,
+      "epoch": 1.6844444444444444,
+      "step": 190
+    },
+    {
+      "loss": 0.5808730602264405,
+      "grad_norm": 0.4427124261856079,
+      "learning_rate": 9.688581314878892e-05,
+      "epoch": 1.7733333333333334,
+      "step": 200
+    },
+    {
+      "loss": 0.6366941928863525,
+      "grad_norm": 0.577314555644989,
+      "learning_rate": 8.996539792387543e-05,
+      "epoch": 1.8622222222222222,
+      "step": 210
+    },
+    {
+      "loss": 0.6232800483703613,
+      "grad_norm": 0.3736749589443207,
+      "learning_rate": 8.304498269896193e-05,
+      "epoch": 1.951111111111111,
+      "step": 220
+    },
+    {
+      "eval_loss": 1.0371954441070557,
+      "eval_runtime": 60.0347,
+      "eval_samples_per_second": 1.666,
+      "eval_steps_per_second": 1.666,
+      "epoch": 2.0,
+      "step": 226
+    },
+    {
+      "loss": 0.6596468448638916,
+      "grad_norm": 0.43450990319252014,
+      "learning_rate": 7.612456747404844e-05,
+      "epoch": 2.0355555555555553,
+      "step": 230
+    },
+    {
+      "loss": 0.5175569534301758,
+      "grad_norm": 0.5357421040534973,
+      "learning_rate": 6.920415224913494e-05,
+      "epoch": 2.1244444444444444,
+      "step": 240
+    },
+    {
+      "loss": 0.49358110427856444,
+      "grad_norm": 0.4734697639942169,
+      "learning_rate": 6.228373702422145e-05,
+      "epoch": 2.2133333333333334,
+      "step": 250
+    },
+    {
+      "loss": 0.43214111328125,
+      "grad_norm": 0.526138961315155,
+      "learning_rate": 5.536332179930796e-05,
+      "epoch": 2.3022222222222224,
+      "step": 260
+    },
+    {
+      "loss": 0.48288736343383787,
+      "grad_norm": 0.7089276909828186,
+      "learning_rate": 4.844290657439446e-05,
+      "epoch": 2.391111111111111,
+      "step": 270
+    },
+    {
+      "loss": 0.4654980659484863,
+      "grad_norm": 0.6069750189781189,
+      "learning_rate": 4.1522491349480966e-05,
+      "epoch": 2.48,
+      "step": 280
+    },
+    {
+      "loss": 0.4116006851196289,
+      "grad_norm": 0.5808397531509399,
+      "learning_rate": 3.460207612456747e-05,
+      "epoch": 2.568888888888889,
+      "step": 290
+    },
+    {
+      "loss": 0.5065722942352295,
+      "grad_norm": 0.4769459664821625,
+      "learning_rate": 2.768166089965398e-05,
+      "epoch": 2.6577777777777776,
+      "step": 300
+    },
+    {
+      "loss": 0.5629632472991943,
+      "grad_norm": 0.5157060623168945,
+      "learning_rate": 2.0761245674740483e-05,
+      "epoch": 2.7466666666666666,
+      "step": 310
+    },
+    {
+      "loss": 0.4408546447753906,
+      "grad_norm": 0.4588596224784851,
+      "learning_rate": 1.384083044982699e-05,
+      "epoch": 2.8355555555555556,
+      "step": 320
+    },
+    {
+      "loss": 0.42911977767944337,
+      "grad_norm": 0.5459583401679993,
+      "learning_rate": 6.920415224913495e-06,
+      "epoch": 2.924444444444444,
+      "step": 330
+    },
+    {
+      "eval_loss": 1.0826495885849,
+      "eval_runtime": 60.1091,
+      "eval_samples_per_second": 1.664,
+      "eval_steps_per_second": 1.664,
+      "epoch": 3.0,
+      "step": 339
+    },
+    {
+      "train_runtime": 8240.7844,
+      "train_samples_per_second": 0.328,
+      "train_steps_per_second": 0.041,
+      "total_flos": 2.2033485759740314e+17,
+      "train_loss": 0.6408278133313564,
+      "epoch": 3.0,
+      "step": 339
+    }
+  ],
+  "lib_versions": {
+    "torch": "2.14.0+cu130",
+    "cuda": "13.0",
+    "transformers": "5.16.1",
+    "peft": "0.20.0",
+    "bitsandbytes": "0.50.2"
+  }
+}