Text Generation
PEFT
Safetensors
lora
qwen3.5
instruction-tuning
File size: 10,659 Bytes
0d413dc
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
W1005 15:56:09.212000 262432 .venv\Lib\site-packages\torch\distributed\elastic\multiprocessing\redirects.py:29] NOTE: Redirects are currently not supported in Windows or MacOs.

Loading weights:   0%|          | 0/320 [00:00<?, ?it/s]
Loading weights:   4%|โ–         | 14/320 [00:00<00:02, 111.46it/s]
Loading weights:   8%|โ–Š         | 26/320 [00:00<00:03, 90.91it/s] 
Loading weights:  11%|โ–ˆโ–        | 36/320 [00:00<00:03, 84.32it/s]
Loading weights:  14%|โ–ˆโ–        | 45/320 [00:00<00:04, 56.70it/s]
Loading weights:  17%|โ–ˆโ–‹        | 55/320 [00:00<00:04, 57.28it/s]
Loading weights:  19%|โ–ˆโ–‰        | 62/320 [00:00<00:04, 58.97it/s]
Loading weights:  22%|โ–ˆโ–ˆโ–       | 69/320 [00:01<00:05, 48.67it/s]
Loading weights:  23%|โ–ˆโ–ˆโ–Ž       | 75/320 [00:01<00:05, 44.12it/s]
Loading weights:  28%|โ–ˆโ–ˆโ–Š       | 88/320 [00:01<00:03, 60.66it/s]
Loading weights:  30%|โ–ˆโ–ˆโ–‰       | 95/320 [00:01<00:04, 53.09it/s]
Loading weights:  33%|โ–ˆโ–ˆโ–ˆโ–Ž      | 105/320 [00:01<00:03, 62.28it/s]
Loading weights:  35%|โ–ˆโ–ˆโ–ˆโ–Œ      | 113/320 [00:01<00:03, 54.97it/s]
Loading weights:  38%|โ–ˆโ–ˆโ–ˆโ–Š      | 120/320 [00:02<00:04, 47.49it/s]
Loading weights:  39%|โ–ˆโ–ˆโ–ˆโ–‰      | 126/320 [00:02<00:05, 37.02it/s]
Loading weights:  41%|โ–ˆโ–ˆโ–ˆโ–ˆโ–     | 132/320 [00:02<00:05, 37.51it/s]
Loading weights:  44%|โ–ˆโ–ˆโ–ˆโ–ˆโ–     | 140/320 [00:02<00:03, 45.40it/s]
Loading weights:  47%|โ–ˆโ–ˆโ–ˆโ–ˆโ–‹     | 149/320 [00:02<00:03, 53.78it/s]
Loading weights:  49%|โ–ˆโ–ˆโ–ˆโ–ˆโ–‰     | 156/320 [00:02<00:02, 54.99it/s]
Loading weights:  51%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–    | 164/320 [00:03<00:03, 47.23it/s]
Loading weights:  56%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ    | 179/320 [00:03<00:02, 67.61it/s]
Loading weights:  58%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Š    | 187/320 [00:03<00:02, 59.27it/s]
Loading weights:  61%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ    | 194/320 [00:03<00:02, 58.84it/s]
Loading weights:  63%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž   | 201/320 [00:03<00:01, 60.39it/s]
Loading weights:  66%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ   | 210/320 [00:03<00:02, 54.00it/s]
Loading weights:  69%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰   | 220/320 [00:03<00:01, 62.38it/s]
Loading weights:  71%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ   | 227/320 [00:04<00:01, 52.24it/s]
Loading weights:  73%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Ž  | 233/320 [00:04<00:01, 45.88it/s]
Loading weights:  77%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‹  | 245/320 [00:04<00:01, 57.74it/s]
Loading weights:  79%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‰  | 252/320 [00:04<00:01, 51.37it/s]
Loading weights:  81%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ  | 258/320 [00:04<00:01, 44.11it/s]
Loading weights:  82%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ– | 263/320 [00:04<00:01, 44.74it/s]
Loading weights:  84%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ– | 268/320 [00:05<00:01, 34.71it/s]
Loading weights:  85%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–Œ | 272/320 [00:05<00:01, 32.85it/s]
Loading weights: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 320/320 [00:05<00:00, 59.63it/s]
model Qwen/Qwen3.5-2B-Base | LoRA rank 8 scale 20.0, 8.4M trainable
train 2400 chats (0 skipped), valid 200 (0 skipped)
1 epochs x 300 optimizer steps of 8 chats, lr 1e-05

----- probes: before training -----
[transformers] `causal_conv1d_fn` is falling back to its reference PyTorch implementation because `causal_conv1d` is not installed. This is correct but much slower; install `causal_conv1d` for the optimized kernel.
[transformers] `causal_conv1d_update` is falling back to its reference PyTorch implementation because `causal_conv1d` is not installed. This is correct but much slower; install `causal_conv1d` for the optimized kernel.
[FAIL] Name three fruits. Write your entire answer in capital letters.
       -> BANANA, ORANGE, APPLE </think> BANANA, ORANGE, APPLE
[PASS] What is the capital of Japan? Reply with JSON containing a single key "capital" and not...
       -> {"capital":"Tokyo"}

[PASS] Is the Pacific the largest ocean on Earth? Answer with exactly one word: yes or no.

       -> No

[FAIL] Write one sentence about the moon that ends with the word "tonight".

       -> The moon is a beautiful, silver disc that rises and sets every night, and tonight, it will be a particularly clear and bright sight in the sky.

[PASS] List two primary colors as a bulleted list where every line starts with '- ', and write...

       -> - Red - Blue

[PASS] What is 7 plus 5? Reply with only the number.

       -> 12

[PASS] Write the word hello in all lowercase letters and nothing else.

       -> hello

[PASS] Describe a cat in fewer than 15 words.

       -> A sleek, black-and-white feline with a tufted tail and a mischievous grin.

[PASS] Give exactly three words that describe the ocean, separated by commas, and nothing else.

       -> Ocean, water, sea

[PASS] Name three planets as a numbered list in the form '1. ...', '2. ...', '3. ...', and wri...

       -> 1. Mercury 2. Venus 3. Earth

[FAIL] Why is the sky blue? Answer in exactly two sentences.

       -> The sky is blue because sunlight scatters more efficiently in shorter blue wavelengths when passing through Earth's atmosphere.

[PASS] Translate 'good morning' into French. Reply with the translation only.

       -> Bonjour

[PASS] Write one sentence that uses both the word 'river' and the word 'lantern'.

       -> The lantern hung from the rafters above the river, casting a warm glow on the water's surface.

[PASS] Start your reply with the word 'Certainly' and then name one color.

       -> Certainly, blue.

[PASS] What is the largest planet in the solar system? Wrap your entire answer in double quota...

       -> "Jupiter"

[FAIL] Describe a sunset in two sentences without using any commas.

       -> The sun dips below the horizon painting the sky in vibrant shades of orange and pink.

[PASS] Give a title for a story about a lost dog, wrapped in double angle brackets like <<titl...

       -> <<A Lost Dog's Journey to Home>>

[PASS] ่ฏทๅช็”จไธ€ไธช่ฏๅ›ž็ญ”๏ผšๆ™ดๅคฉๆ—ถๅคฉ็ฉบ้€šๅธธๆ˜ฏไป€ไนˆ้ขœ่‰ฒ๏ผŸ

       -> ่“่‰ฒ

[PASS] ็”จไธญๆ–‡ๅˆ—ๅ‡บไธ‰็งๆฐดๆžœ๏ผŒๆฏ่กŒไธ€็ง๏ผŒไธ่ฆๅ†™ๅ…ถไป–ๅ†…ๅฎนใ€‚

       -> ่‹นๆžœ ้ฆ™่•‰ ๆฉ™ๅญ

----- instructions 15/19 -----



step 0 | valid loss 1.361

[transformers] `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.

epoch 1 step 25/300 | train loss 1.259 | lr 9.99e-06 | grad norm 8.46 (max 11.81) | 1450 reply tok/s | peak 10.4 GB

epoch 1 step 50/300 | train loss 1.292 | lr 9.75e-06 | grad norm 7.67 (max 11.88) | 1535 reply tok/s | peak 11.0 GB

epoch 1 step 75/300 | train loss 1.236 | lr 9.17e-06 | grad norm 7.66 (max 11.16) | 1544 reply tok/s | peak 11.0 GB

epoch 1 step 100/300 | train loss 1.212 | lr 8.31e-06 | grad norm 7.98 (max 11.08) | 1375 reply tok/s | peak 11.0 GB

step 100 | valid loss 1.259 (best 1.259)

epoch 1 step 125/300 | train loss 1.254 | lr 7.22e-06 | grad norm 8.12 (max 11.79) | 1297 reply tok/s | peak 11.4 GB

epoch 1 step 150/300 | train loss 1.216 | lr 6.00e-06 | grad norm 7.74 (max 9.36) | 1401 reply tok/s | peak 11.4 GB

epoch 1 step 175/300 | train loss 1.249 | lr 4.75e-06 | grad norm 7.49 (max 12.50) | 1584 reply tok/s | peak 11.4 GB

epoch 1 step 200/300 | train loss 1.241 | lr 3.55e-06 | grad norm 7.39 (max 11.12) | 1579 reply tok/s | peak 11.5 GB

step 200 | valid loss 1.252 (best 1.252)

epoch 1 step 225/300 | train loss 1.199 | lr 2.50e-06 | grad norm 7.48 (max 9.20) | 1262 reply tok/s | peak 11.5 GB

epoch 1 step 250/300 | train loss 1.200 | lr 1.69e-06 | grad norm 7.58 (max 12.96) | 1504 reply tok/s | peak 11.5 GB

epoch 1 step 275/300 | train loss 1.156 | lr 1.18e-06 | grad norm 7.89 (max 11.22) | 1443 reply tok/s | peak 11.5 GB

epoch 1 step 300/300 | train loss 1.165 | lr 1.00e-06 | grad norm 7.67 (max 10.73) | 1441 reply tok/s | peak 11.5 GB

step 300 | valid loss 1.250 (best 1.250)

== end of epoch 1 | valid loss 1.250 (best 1.250)

C:\Users\arisp\Documents\Research\Memorize\.venv\Lib\site-packages\peft\utils\save_and_load.py:313: UserWarning: Could not find a config file in Qwen/Qwen3.5-2B-Base - will assume that the vocabulary was not modified.

  warnings.warn(



----- probes: after epoch 1 -----

[PASS] Name three fruits. Write your entire answer in capital letters.

       -> BANANA, APPLE, ORANGE

[PASS] What is the capital of Japan? Reply with JSON containing a single key "capital" and not...

       -> ```json { "capital": "Tokyo" } ```

[PASS] Is the Pacific the largest ocean on Earth? Answer with exactly one word: yes or no.

       -> Yes

[FAIL] Write one sentence about the moon that ends with the word "tonight".

       -> Tonight, the moon casts a soft glow over the landscape, illuminating the night sky with its gentle light.

[PASS] List two primary colors as a bulleted list where every line starts with '- ', and write...

       -> - Red - Blue

[PASS] What is 7 plus 5? Reply with only the number.

       -> 12

[PASS] Write the word hello in all lowercase letters and nothing else.

       -> hello

[PASS] Describe a cat in fewer than 15 words.

       -> A fluffy feline with a playful spirit.

[PASS] Give exactly three words that describe the ocean, separated by commas, and nothing else.

       -> Ocean, waves, depth

[PASS] Name three planets as a numbered list in the form '1. ...', '2. ...', '3. ...', and wri...

       -> 1. Mercury 2. Venus 3. Earth

[PASS] Why is the sky blue? Answer in exactly two sentences.

       -> The sky appears blue because the sun's light is scattered by the molecules and tiny water droplets in the Earth's atmosphere. This scattering, known as Rayleigh scattering, is more effective for shorter wavelengths of...

[PASS] Translate 'good morning' into French. Reply with the translation only.

       -> Bonjour

[PASS] Write one sentence that uses both the word 'river' and the word 'lantern'.

       -> The lantern hung over the river as the moonlight shone through the water.

[PASS] Start your reply with the word 'Certainly' and then name one color.

       -> Certainly, blue.

[PASS] What is the largest planet in the solar system? Wrap your entire answer in double quota...

       -> "Jupiter"

[PASS] Describe a sunset in two sentences without using any commas.

       -> The sun dips below the horizon casting a golden glow across the sky. The colors blend into a beautiful tapestry of orange and pink hues.

[PASS] Give a title for a story about a lost dog, wrapped in double angle brackets like <<titl...

       -> <<A Lost Dog's Journey>>

[PASS] ่ฏทๅช็”จไธ€ไธช่ฏๅ›ž็ญ”๏ผšๆ™ดๅคฉๆ—ถๅคฉ็ฉบ้€šๅธธๆ˜ฏไป€ไนˆ้ขœ่‰ฒ๏ผŸ

       -> ่“่‰ฒ

[PASS] ็”จไธญๆ–‡ๅˆ—ๅ‡บไธ‰็งๆฐดๆžœ๏ผŒๆฏ่กŒไธ€็ง๏ผŒไธ่ฆๅ†™ๅ…ถไป–ๅ†…ๅฎนใ€‚

       -> ่‹นๆžœ ้ฆ™่•‰ ๆฉ™ๅญ

----- instructions 18/19 -----



saved adapter to adapters\sft-run2-torch