File size: 62,485 Bytes
7d5fc54 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | Valid precision + execution provider combinations are: FP32 CPU, FP32 CUDA, FP16 CUDA, FP16 DML, FP16 TRT-RTX, BF16 CUDA, BF16 TRT-RTX, INT8 CPU, INT8 CUDA, INT8 WebGPU, INT4 CPU, INT4 CUDA, INT4 DML, INT4 WebGPU
Extra options: {}
GroupQueryAttention (GQA) is used in this model.
GroupQueryAttention (GQA) is used in this model.
Loading weights: 0%| | 0/426 [00:00<?, ?it/s]
Loading weights: 16%|█▌ | 69/426 [00:00<00:00, 672.81it/s]
Loading weights: 34%|███▍ | 144/426 [00:00<00:00, 715.00it/s]
Loading weights: 52%|█████▏ | 221/426 [00:00<00:00, 735.44it/s]
Loading weights: 69%|██████▉ | 296/426 [00:00<00:00, 719.97it/s]
Loading weights: 89%|████████▉ | 379/426 [00:00<00:00, 754.67it/s]
Loading weights: 100%|██████████| 426/426 [00:00<00:00, 754.59it/s]
Reading embedding layer
Reading layer 0
Reading layer 1
Reading layer 2
Reading layer 3
Reading layer 4
Reading layer 5
Reading layer 6
Reading layer 7
Reading layer 8
Reading layer 9
Reading layer 10
Reading layer 11
Reading layer 12
Reading layer 13
Reading layer 14
Reading layer 15
Reading layer 16
Reading layer 17
Reading layer 18
Reading layer 19
Reading layer 20
Reading layer 21
Reading layer 22
Reading layer 23
Reading layer 24
Reading layer 25
Reading layer 26
Reading layer 27
Reading layer 28
Reading layer 29
Reading layer 30
Reading layer 31
Reading final norm
Reading LM head
Building MTP (multi-token prediction) head -> mtp.onnx
Saving ONNX model in /home/anon/knowline/onnx/model
0it [00:00, ?it/s]
Saving model.layers.0.input_layernorm.weight (f16, [2560]): 0%| | 1/404 [00:00<00:01, 257.64it/s]
Saving model.layers.0.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 0%| | 2/404 [00:00<00:00, 479.76it/s]
Saving model.layers.0.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 1%| | 3/404 [00:00<00:03, 102.33it/s]
Saving model.layers.0.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 1%| | 4/404 [00:00<00:04, 92.50it/s]
Saving model.layers.0.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 1%| | 5/404 [00:00<00:03, 114.79it/s]
Saving model.layers.0.linear_attn.conv1d.weight (f16, [8192,1,4]): 1%|▏ | 6/404 [00:00<00:02, 137.16it/s]
Saving model.layers.0.linear_attn.conv1d.bias (f16, [8192]): 2%|▏ | 7/404 [00:00<00:02, 159.58it/s]
Saving model.layers.0.linear_attn.A_log (f32, [32]): 2%|▏ | 8/404 [00:00<00:02, 181.96it/s]
Saving model.layers.0.linear_attn.dt_bias (f32, [32]): 2%|▏ | 9/404 [00:00<00:01, 204.30it/s]
Saving model.layers.0.linear_attn.norm.weight (f16, [128]): 2%|▏ | 10/404 [00:00<00:01, 226.72it/s]
Saving model.layers.0.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 3%|▎ | 11/404 [00:00<00:01, 249.07it/s]
Saving model.layers.0.post_attention_layernorm.weight (f16, [2560]): 3%|▎ | 12/404 [00:00<00:01, 208.07it/s]
Saving model.layers.0.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 3%|▎ | 13/404 [00:00<00:01, 224.36it/s]
Saving model.layers.0.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 3%|▎ | 14/404 [00:00<00:13, 29.58it/s]
Saving model.layers.0.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 3%|▎ | 14/404 [00:00<00:13, 29.58it/s]
Saving model.layers.1.input_layernorm.weight (f16, [2560]): 4%|▎ | 15/404 [00:00<00:13, 29.58it/s]
Saving model.layers.1.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 4%|▍ | 16/404 [00:00<00:13, 29.58it/s]
Saving model.layers.1.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 4%|▍ | 17/404 [00:00<00:13, 29.58it/s]
Saving model.layers.1.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 4%|▍ | 18/404 [00:00<00:13, 29.58it/s]
Saving model.layers.1.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 5%|▍ | 19/404 [00:00<00:13, 29.58it/s]
Saving model.layers.1.linear_attn.conv1d.weight (f16, [8192,1,4]): 5%|▍ | 20/404 [00:00<00:12, 29.58it/s]
Saving model.layers.1.linear_attn.conv1d.bias (f16, [8192]): 5%|▌ | 21/404 [00:00<00:12, 29.58it/s]
Saving model.layers.1.linear_attn.A_log (f32, [32]): 5%|▌ | 22/404 [00:00<00:12, 29.58it/s]
Saving model.layers.1.linear_attn.dt_bias (f32, [32]): 6%|▌ | 23/404 [00:00<00:12, 29.58it/s]
Saving model.layers.1.linear_attn.norm.weight (f16, [128]): 6%|▌ | 24/404 [00:00<00:12, 29.58it/s]
Saving model.layers.1.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 6%|▌ | 25/404 [00:00<00:12, 29.58it/s]
Saving model.layers.1.post_attention_layernorm.weight (f16, [2560]): 6%|▋ | 26/404 [00:00<00:12, 29.58it/s]
Saving model.layers.1.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 7%|▋ | 27/404 [00:00<00:12, 29.58it/s]
Saving model.layers.1.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 7%|▋ | 28/404 [00:00<00:13, 28.42it/s]
Saving model.layers.1.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 7%|▋ | 28/404 [00:00<00:13, 28.42it/s]
Saving model.layers.2.input_layernorm.weight (f16, [2560]): 7%|▋ | 29/404 [00:01<00:13, 28.42it/s]
Saving model.layers.2.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 7%|▋ | 30/404 [00:01<00:13, 28.42it/s]
Saving model.layers.2.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 8%|▊ | 31/404 [00:01<00:13, 28.42it/s]
Saving model.layers.2.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 8%|▊ | 32/404 [00:01<00:13, 28.42it/s]
Saving model.layers.2.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 8%|▊ | 33/404 [00:01<00:13, 28.42it/s]
Saving model.layers.2.linear_attn.conv1d.weight (f16, [8192,1,4]): 8%|▊ | 34/404 [00:01<00:13, 28.42it/s]
Saving model.layers.2.linear_attn.conv1d.bias (f16, [8192]): 9%|▊ | 35/404 [00:01<00:12, 28.42it/s]
Saving model.layers.2.linear_attn.A_log (f32, [32]): 9%|▉ | 36/404 [00:01<00:12, 28.42it/s]
Saving model.layers.2.linear_attn.dt_bias (f32, [32]): 9%|▉ | 37/404 [00:01<00:12, 28.42it/s]
Saving model.layers.2.linear_attn.norm.weight (f16, [128]): 9%|▉ | 38/404 [00:01<00:12, 28.42it/s]
Saving model.layers.2.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 10%|▉ | 39/404 [00:01<00:12, 28.42it/s]
Saving model.layers.2.post_attention_layernorm.weight (f16, [2560]): 10%|▉ | 40/404 [00:01<00:12, 28.42it/s]
Saving model.layers.2.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 10%|█ | 41/404 [00:01<00:12, 28.42it/s]
Saving model.layers.2.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 10%|█ | 42/404 [00:01<00:12, 28.04it/s]
Saving model.layers.2.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 10%|█ | 42/404 [00:01<00:12, 28.04it/s]
Saving model.layers.3.input_layernorm.weight (f16, [2560]): 11%|█ | 43/404 [00:01<00:12, 28.04it/s]
Saving model.layers.3.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 11%|█ | 44/404 [00:01<00:12, 28.04it/s]
Saving model.layers.3.attn.q_norm.layernorm.weight (f16, [256]): 11%|█ | 45/404 [00:01<00:12, 28.04it/s]
Saving model.layers.3.attn.k_norm.layernorm.weight (f16, [256]): 11%|█▏ | 46/404 [00:01<00:12, 28.04it/s]
Saving cos_cache (f32, [262144,32]): 12%|█▏ | 47/404 [00:01<00:12, 28.04it/s]
Saving sin_cache (f32, [262144,32]): 12%|█▏ | 48/404 [00:01<00:12, 28.04it/s]
Saving model.layers.3.attn.o_proj.MatMul.weight (f16, [4096,2560]): 12%|█▏ | 49/404 [00:01<00:12, 28.04it/s]
Saving model.layers.3.post_attention_layernorm.weight (f16, [2560]): 12%|█▏ | 50/404 [00:01<00:12, 28.04it/s]
Saving model.layers.3.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 13%|█▎ | 51/404 [00:01<00:12, 28.04it/s]
Saving model.layers.3.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 13%|█▎ | 52/404 [00:01<00:14, 24.68it/s]
Saving model.layers.3.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 13%|█▎ | 52/404 [00:01<00:14, 24.68it/s]
Saving model.layers.4.input_layernorm.weight (f16, [2560]): 13%|█▎ | 53/404 [00:02<00:14, 24.68it/s]
Saving model.layers.4.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 13%|█▎ | 54/404 [00:02<00:14, 24.68it/s]
Saving model.layers.4.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 14%|█▎ | 55/404 [00:02<00:14, 24.68it/s]
Saving model.layers.4.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 14%|█▍ | 56/404 [00:02<00:14, 24.68it/s]
Saving model.layers.4.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 14%|█▍ | 57/404 [00:02<00:14, 24.68it/s]
Saving model.layers.4.linear_attn.conv1d.weight (f16, [8192,1,4]): 14%|█▍ | 58/404 [00:02<00:14, 24.68it/s]
Saving model.layers.4.linear_attn.conv1d.bias (f16, [8192]): 15%|█▍ | 59/404 [00:02<00:13, 24.68it/s]
Saving model.layers.4.linear_attn.A_log (f32, [32]): 15%|█▍ | 60/404 [00:02<00:13, 24.68it/s]
Saving model.layers.4.linear_attn.dt_bias (f32, [32]): 15%|█▌ | 61/404 [00:02<00:13, 24.68it/s]
Saving model.layers.4.linear_attn.norm.weight (f16, [128]): 15%|█▌ | 62/404 [00:02<00:13, 24.68it/s]
Saving model.layers.4.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 16%|█▌ | 63/404 [00:02<00:13, 24.68it/s]
Saving model.layers.4.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 16%|█▌ | 64/404 [00:02<00:09, 34.15it/s]
Saving model.layers.4.post_attention_layernorm.weight (f16, [2560]): 16%|█▌ | 64/404 [00:02<00:09, 34.15it/s]
Saving model.layers.4.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 16%|█▌ | 65/404 [00:02<00:09, 34.15it/s]
Saving model.layers.4.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 16%|█▋ | 66/404 [00:02<00:09, 34.15it/s]
Saving model.layers.5.input_layernorm.weight (f16, [2560]): 17%|█▋ | 67/404 [00:02<00:09, 34.15it/s]
Saving model.layers.5.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 17%|█▋ | 68/404 [00:02<00:09, 34.15it/s]
Saving model.layers.5.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 17%|█▋ | 69/404 [00:02<00:09, 34.15it/s]
Saving model.layers.5.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 17%|█▋ | 70/404 [00:02<00:13, 25.32it/s]
Saving model.layers.5.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 17%|█▋ | 70/404 [00:02<00:13, 25.32it/s]
Saving model.layers.5.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 18%|█▊ | 71/404 [00:02<00:13, 25.32it/s]
Saving model.layers.5.linear_attn.conv1d.weight (f16, [8192,1,4]): 18%|█▊ | 72/404 [00:02<00:13, 25.32it/s]
Saving model.layers.5.linear_attn.conv1d.bias (f16, [8192]): 18%|█▊ | 73/404 [00:02<00:13, 25.32it/s]
Saving model.layers.5.linear_attn.A_log (f32, [32]): 18%|█▊ | 74/404 [00:02<00:13, 25.32it/s]
Saving model.layers.5.linear_attn.dt_bias (f32, [32]): 19%|█▊ | 75/404 [00:02<00:12, 25.32it/s]
Saving model.layers.5.linear_attn.norm.weight (f16, [128]): 19%|█▉ | 76/404 [00:02<00:12, 25.32it/s]
Saving model.layers.5.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 19%|█▉ | 77/404 [00:02<00:12, 25.32it/s]
Saving model.layers.5.post_attention_layernorm.weight (f16, [2560]): 19%|█▉ | 78/404 [00:02<00:12, 25.32it/s]
Saving model.layers.5.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 20%|█▉ | 79/404 [00:02<00:12, 25.32it/s]
Saving model.layers.5.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 20%|█▉ | 80/404 [00:03<00:13, 24.43it/s]
Saving model.layers.5.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 20%|█▉ | 80/404 [00:03<00:13, 24.43it/s]
Saving model.layers.6.input_layernorm.weight (f16, [2560]): 20%|██ | 81/404 [00:03<00:13, 24.43it/s]
Saving model.layers.6.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 20%|██ | 82/404 [00:03<00:13, 24.43it/s]
Saving model.layers.6.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 21%|██ | 83/404 [00:03<00:13, 24.43it/s]
Saving model.layers.6.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 21%|██ | 84/404 [00:03<00:13, 24.43it/s]
Saving model.layers.6.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 21%|██ | 85/404 [00:03<00:13, 24.43it/s]
Saving model.layers.6.linear_attn.conv1d.weight (f16, [8192,1,4]): 21%|██▏ | 86/404 [00:03<00:13, 24.43it/s]
Saving model.layers.6.linear_attn.conv1d.bias (f16, [8192]): 22%|██▏ | 87/404 [00:03<00:12, 24.43it/s]
Saving model.layers.6.linear_attn.A_log (f32, [32]): 22%|██▏ | 88/404 [00:03<00:12, 24.43it/s]
Saving model.layers.6.linear_attn.dt_bias (f32, [32]): 22%|██▏ | 89/404 [00:03<00:12, 24.43it/s]
Saving model.layers.6.linear_attn.norm.weight (f16, [128]): 22%|██▏ | 90/404 [00:03<00:12, 24.43it/s]
Saving model.layers.6.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 23%|██▎ | 91/404 [00:03<00:12, 24.43it/s]
Saving model.layers.6.post_attention_layernorm.weight (f16, [2560]): 23%|██▎ | 92/404 [00:03<00:12, 24.43it/s]
Saving model.layers.6.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 23%|██▎ | 93/404 [00:03<00:12, 24.43it/s]
Saving model.layers.6.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 23%|██▎ | 94/404 [00:03<00:12, 25.59it/s]
Saving model.layers.6.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 23%|██▎ | 94/404 [00:03<00:12, 25.59it/s]
Saving model.layers.7.input_layernorm.weight (f16, [2560]): 24%|██▎ | 95/404 [00:03<00:12, 25.59it/s]
Saving model.layers.7.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 24%|██▍ | 96/404 [00:03<00:12, 25.59it/s]
Saving model.layers.7.attn.q_norm.layernorm.weight (f16, [256]): 24%|██▍ | 97/404 [00:03<00:11, 25.59it/s]
Saving model.layers.7.attn.k_norm.layernorm.weight (f16, [256]): 24%|██▍ | 98/404 [00:03<00:11, 25.59it/s]
Saving model.layers.7.attn.o_proj.MatMul.weight (f16, [4096,2560]): 25%|██▍ | 99/404 [00:03<00:11, 25.59it/s]
Saving model.layers.7.post_attention_layernorm.weight (f16, [2560]): 25%|██▍ | 100/404 [00:03<00:11, 25.59it/s]
Saving model.layers.7.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 25%|██▌ | 101/404 [00:03<00:11, 25.59it/s]
Saving model.layers.7.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 25%|██▌ | 102/404 [00:04<00:13, 22.30it/s]
Saving model.layers.7.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 25%|██▌ | 102/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.input_layernorm.weight (f16, [2560]): 25%|██▌ | 103/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 26%|██▌ | 104/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 26%|██▌ | 105/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 26%|██▌ | 106/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 26%|██▋ | 107/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.conv1d.weight (f16, [8192,1,4]): 27%|██▋ | 108/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.conv1d.bias (f16, [8192]): 27%|██▋ | 109/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.A_log (f32, [32]): 27%|██▋ | 110/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.dt_bias (f32, [32]): 27%|██▋ | 111/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.norm.weight (f16, [128]): 28%|██▊ | 112/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 28%|██▊ | 113/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.post_attention_layernorm.weight (f16, [2560]): 28%|██▊ | 114/404 [00:04<00:13, 22.30it/s]
Saving model.layers.8.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 28%|██▊ | 115/404 [00:04<00:12, 22.30it/s]
Saving model.layers.8.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 29%|██▊ | 116/404 [00:04<00:11, 24.11it/s]
Saving model.layers.8.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 29%|██▊ | 116/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.input_layernorm.weight (f16, [2560]): 29%|██▉ | 117/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 29%|██▉ | 118/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 29%|██▉ | 119/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 30%|██▉ | 120/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 30%|██▉ | 121/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.conv1d.weight (f16, [8192,1,4]): 30%|███ | 122/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.conv1d.bias (f16, [8192]): 30%|███ | 123/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.A_log (f32, [32]): 31%|███ | 124/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.dt_bias (f32, [32]): 31%|███ | 125/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.norm.weight (f16, [128]): 31%|███ | 126/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 31%|███▏ | 127/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.post_attention_layernorm.weight (f16, [2560]): 32%|███▏ | 128/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 32%|███▏ | 129/404 [00:04<00:11, 24.11it/s]
Saving model.layers.9.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 32%|███▏ | 130/404 [00:05<00:10, 25.22it/s]
Saving model.layers.9.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 32%|███▏ | 130/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.input_layernorm.weight (f16, [2560]): 32%|███▏ | 131/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 33%|███▎ | 132/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 33%|███▎ | 133/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 33%|███▎ | 134/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 33%|███▎ | 135/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.conv1d.weight (f16, [8192,1,4]): 34%|███▎ | 136/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.conv1d.bias (f16, [8192]): 34%|███▍ | 137/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.A_log (f32, [32]): 34%|███▍ | 138/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.dt_bias (f32, [32]): 34%|███▍ | 139/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.norm.weight (f16, [128]): 35%|███▍ | 140/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 35%|███▍ | 141/404 [00:05<00:10, 25.22it/s]
Saving model.layers.10.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 35%|███▌ | 142/404 [00:05<00:07, 33.03it/s]
Saving model.layers.10.post_attention_layernorm.weight (f16, [2560]): 35%|███▌ | 142/404 [00:05<00:07, 33.03it/s]
Saving model.layers.10.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 35%|███▌ | 143/404 [00:05<00:07, 33.03it/s]
Saving model.layers.10.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 36%|███▌ | 144/404 [00:05<00:07, 33.03it/s]
Saving model.layers.11.input_layernorm.weight (f16, [2560]): 36%|███▌ | 145/404 [00:05<00:07, 33.03it/s]
Saving model.layers.11.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 36%|███▌ | 146/404 [00:05<00:07, 33.03it/s]
Saving model.layers.11.attn.q_norm.layernorm.weight (f16, [256]): 36%|███▋ | 147/404 [00:05<00:07, 33.03it/s]
Saving model.layers.11.attn.q_norm.layernorm.weight (f16, [256]): 37%|███▋ | 148/404 [00:05<00:09, 26.06it/s]
Saving model.layers.11.attn.k_norm.layernorm.weight (f16, [256]): 37%|███▋ | 148/404 [00:05<00:09, 26.06it/s]
Saving model.layers.11.attn.o_proj.MatMul.weight (f16, [4096,2560]): 37%|███▋ | 149/404 [00:05<00:09, 26.06it/s]
Saving model.layers.11.post_attention_layernorm.weight (f16, [2560]): 37%|███▋ | 150/404 [00:05<00:09, 26.06it/s]
Saving model.layers.11.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 37%|███▋ | 151/404 [00:05<00:09, 26.06it/s]
Saving model.layers.11.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 38%|███▊ | 152/404 [00:06<00:09, 26.06it/s]
Saving model.layers.11.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 38%|███▊ | 153/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.input_layernorm.weight (f16, [2560]): 38%|███▊ | 153/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 38%|███▊ | 154/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 38%|███▊ | 155/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 39%|███▊ | 156/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 39%|███▉ | 157/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.conv1d.weight (f16, [8192,1,4]): 39%|███▉ | 158/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.conv1d.bias (f16, [8192]): 39%|███▉ | 159/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.A_log (f32, [32]): 40%|███▉ | 160/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.dt_bias (f32, [32]): 40%|███▉ | 161/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.norm.weight (f16, [128]): 40%|████ | 162/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 40%|████ | 163/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.post_attention_layernorm.weight (f16, [2560]): 41%|████ | 164/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 41%|████ | 165/404 [00:06<00:11, 21.15it/s]
Saving model.layers.12.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 41%|████ | 166/404 [00:06<00:10, 23.45it/s]
Saving model.layers.12.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 41%|████ | 166/404 [00:06<00:10, 23.45it/s]
Saving model.layers.13.input_layernorm.weight (f16, [2560]): 41%|████▏ | 167/404 [00:06<00:10, 23.45it/s]
Saving model.layers.13.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 42%|████▏ | 168/404 [00:06<00:10, 23.45it/s]
Saving model.layers.13.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 42%|████▏ | 169/404 [00:06<00:10, 23.45it/s]
Saving model.layers.13.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 42%|████▏ | 170/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 42%|████▏ | 171/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.linear_attn.conv1d.weight (f16, [8192,1,4]): 43%|████▎ | 172/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.linear_attn.conv1d.bias (f16, [8192]): 43%|████▎ | 173/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.linear_attn.A_log (f32, [32]): 43%|████▎ | 174/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.linear_attn.dt_bias (f32, [32]): 43%|████▎ | 175/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.linear_attn.norm.weight (f16, [128]): 44%|████▎ | 176/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 44%|████▍ | 177/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.post_attention_layernorm.weight (f16, [2560]): 44%|████▍ | 178/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 44%|████▍ | 179/404 [00:06<00:09, 23.45it/s]
Saving model.layers.13.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 45%|████▍ | 180/404 [00:06<00:08, 25.09it/s]
Saving model.layers.13.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 45%|████▍ | 180/404 [00:06<00:08, 25.09it/s]
Saving model.layers.14.input_layernorm.weight (f16, [2560]): 45%|████▍ | 181/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 45%|████▌ | 182/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 45%|████▌ | 183/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 46%|████▌ | 184/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 46%|████▌ | 185/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.conv1d.weight (f16, [8192,1,4]): 46%|████▌ | 186/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.conv1d.bias (f16, [8192]): 46%|████▋ | 187/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.A_log (f32, [32]): 47%|████▋ | 188/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.dt_bias (f32, [32]): 47%|████▋ | 189/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.norm.weight (f16, [128]): 47%|████▋ | 190/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 47%|████▋ | 191/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.post_attention_layernorm.weight (f16, [2560]): 48%|████▊ | 192/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 48%|████▊ | 193/404 [00:07<00:08, 25.09it/s]
Saving model.layers.14.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 48%|████▊ | 194/404 [00:07<00:08, 25.86it/s]
Saving model.layers.14.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 48%|████▊ | 194/404 [00:07<00:08, 25.86it/s]
Saving model.layers.15.input_layernorm.weight (f16, [2560]): 48%|████▊ | 195/404 [00:07<00:08, 25.86it/s]
Saving model.layers.15.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 49%|████▊ | 196/404 [00:07<00:08, 25.86it/s]
Saving model.layers.15.attn.q_norm.layernorm.weight (f16, [256]): 49%|████▉ | 197/404 [00:07<00:08, 25.86it/s]
Saving model.layers.15.attn.k_norm.layernorm.weight (f16, [256]): 49%|████▉ | 198/404 [00:07<00:07, 25.86it/s]
Saving model.layers.15.attn.o_proj.MatMul.weight (f16, [4096,2560]): 49%|████▉ | 199/404 [00:07<00:07, 25.86it/s]
Saving model.layers.15.post_attention_layernorm.weight (f16, [2560]): 50%|████▉ | 200/404 [00:07<00:07, 25.86it/s]
Saving model.layers.15.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 50%|████▉ | 201/404 [00:07<00:07, 25.86it/s]
Saving model.layers.15.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 50%|█████ | 202/404 [00:07<00:08, 23.13it/s]
Saving model.layers.15.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 50%|█████ | 202/404 [00:07<00:08, 23.13it/s]
Saving model.layers.16.input_layernorm.weight (f16, [2560]): 50%|█████ | 203/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 50%|█████ | 204/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 51%|█████ | 205/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 51%|█████ | 206/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 51%|█████ | 207/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.conv1d.weight (f16, [8192,1,4]): 51%|█████▏ | 208/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.conv1d.bias (f16, [8192]): 52%|█████▏ | 209/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.A_log (f32, [32]): 52%|█████▏ | 210/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.dt_bias (f32, [32]): 52%|█████▏ | 211/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.norm.weight (f16, [128]): 52%|█████▏ | 212/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 53%|█████▎ | 213/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.post_attention_layernorm.weight (f16, [2560]): 53%|█████▎ | 214/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 53%|█████▎ | 215/404 [00:08<00:08, 23.13it/s]
Saving model.layers.16.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 53%|█████▎ | 216/404 [00:08<00:07, 24.91it/s]
Saving model.layers.16.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 53%|█████▎ | 216/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.input_layernorm.weight (f16, [2560]): 54%|█████▎ | 217/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 54%|█████▍ | 218/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 54%|█████▍ | 219/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 54%|█████▍ | 220/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 55%|█████▍ | 221/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.conv1d.weight (f16, [8192,1,4]): 55%|█████▍ | 222/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.conv1d.bias (f16, [8192]): 55%|█████▌ | 223/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.A_log (f32, [32]): 55%|█████▌ | 224/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.dt_bias (f32, [32]): 56%|█████▌ | 225/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.norm.weight (f16, [128]): 56%|█████▌ | 226/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 56%|█████▌ | 227/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.post_attention_layernorm.weight (f16, [2560]): 56%|█████▋ | 228/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 57%|█████▋ | 229/404 [00:08<00:07, 24.91it/s]
Saving model.layers.17.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 57%|█████▋ | 230/404 [00:08<00:06, 26.16it/s]
Saving model.layers.17.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 57%|█████▋ | 230/404 [00:08<00:06, 26.16it/s]
Saving model.layers.18.input_layernorm.weight (f16, [2560]): 57%|█████▋ | 231/404 [00:08<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 57%|█████▋ | 232/404 [00:08<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 58%|█████▊ | 233/404 [00:08<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 58%|█████▊ | 234/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 58%|█████▊ | 235/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.conv1d.weight (f16, [8192,1,4]): 58%|█████▊ | 236/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.conv1d.bias (f16, [8192]): 59%|█████▊ | 237/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.A_log (f32, [32]): 59%|█████▉ | 238/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.dt_bias (f32, [32]): 59%|█████▉ | 239/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.norm.weight (f16, [128]): 59%|█████▉ | 240/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 60%|█████▉ | 241/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.post_attention_layernorm.weight (f16, [2560]): 60%|█████▉ | 242/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 60%|██████ | 243/404 [00:09<00:06, 26.16it/s]
Saving model.layers.18.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 60%|██████ | 244/404 [00:09<00:05, 27.04it/s]
Saving model.layers.18.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 60%|██████ | 244/404 [00:09<00:05, 27.04it/s]
Saving model.layers.19.input_layernorm.weight (f16, [2560]): 61%|██████ | 245/404 [00:09<00:05, 27.04it/s]
Saving model.layers.19.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 61%|██████ | 246/404 [00:09<00:05, 27.04it/s]
Saving model.layers.19.attn.q_norm.layernorm.weight (f16, [256]): 61%|██████ | 247/404 [00:09<00:05, 27.04it/s]
Saving model.layers.19.attn.k_norm.layernorm.weight (f16, [256]): 61%|██████▏ | 248/404 [00:09<00:05, 27.04it/s]
Saving model.layers.19.attn.o_proj.MatMul.weight (f16, [4096,2560]): 62%|██████▏ | 249/404 [00:09<00:05, 27.04it/s]
Saving model.layers.19.post_attention_layernorm.weight (f16, [2560]): 62%|██████▏ | 250/404 [00:09<00:05, 27.04it/s]
Saving model.layers.19.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 62%|██████▏ | 251/404 [00:09<00:05, 27.04it/s]
Saving model.layers.19.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 62%|██████▏ | 252/404 [00:09<00:06, 24.13it/s]
Saving model.layers.19.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 62%|██████▏ | 252/404 [00:09<00:06, 24.13it/s]
Saving model.layers.20.input_layernorm.weight (f16, [2560]): 63%|██████▎ | 253/404 [00:09<00:06, 24.13it/s]
Saving model.layers.20.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 63%|██████▎ | 254/404 [00:09<00:06, 24.13it/s]
Saving model.layers.20.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 63%|██████▎ | 255/404 [00:09<00:06, 24.13it/s]
Saving model.layers.20.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 63%|██████▎ | 256/404 [00:09<00:06, 24.13it/s]
Saving model.layers.20.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 64%|██████▎ | 257/404 [00:09<00:06, 24.13it/s]
Saving model.layers.20.linear_attn.conv1d.weight (f16, [8192,1,4]): 64%|██████▍ | 258/404 [00:09<00:06, 24.13it/s]
Saving model.layers.20.linear_attn.conv1d.bias (f16, [8192]): 64%|██████▍ | 259/404 [00:09<00:06, 24.13it/s]
Saving model.layers.20.linear_attn.A_log (f32, [32]): 64%|██████▍ | 260/404 [00:09<00:05, 24.13it/s]
Saving model.layers.20.linear_attn.dt_bias (f32, [32]): 65%|██████▍ | 261/404 [00:09<00:05, 24.13it/s]
Saving model.layers.20.linear_attn.norm.weight (f16, [128]): 65%|██████▍ | 262/404 [00:09<00:05, 24.13it/s]
Saving model.layers.20.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 65%|██████▌ | 263/404 [00:09<00:05, 24.13it/s]
Saving model.layers.20.post_attention_layernorm.weight (f16, [2560]): 65%|██████▌ | 264/404 [00:09<00:05, 24.13it/s]
Saving model.layers.20.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 66%|██████▌ | 265/404 [00:09<00:05, 24.13it/s]
Saving model.layers.20.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 66%|██████▌ | 266/404 [00:10<00:05, 25.38it/s]
Saving model.layers.20.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 66%|██████▌ | 266/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.input_layernorm.weight (f16, [2560]): 66%|██████▌ | 267/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 66%|██████▋ | 268/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 67%|██████▋ | 269/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 67%|██████▋ | 270/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 67%|██████▋ | 271/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.conv1d.weight (f16, [8192,1,4]): 67%|██████▋ | 272/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.conv1d.bias (f16, [8192]): 68%|██████▊ | 273/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.A_log (f32, [32]): 68%|██████▊ | 274/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.dt_bias (f32, [32]): 68%|██████▊ | 275/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.norm.weight (f16, [128]): 68%|██████▊ | 276/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 69%|██████▊ | 277/404 [00:10<00:05, 25.38it/s]
Saving model.layers.21.post_attention_layernorm.weight (f16, [2560]): 69%|██████▉ | 278/404 [00:10<00:04, 25.38it/s]
Saving model.layers.21.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 69%|██████▉ | 279/404 [00:10<00:04, 25.38it/s]
Saving model.layers.21.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 69%|██████▉ | 280/404 [00:10<00:04, 26.50it/s]
Saving model.layers.21.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 69%|██████▉ | 280/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.input_layernorm.weight (f16, [2560]): 70%|██████▉ | 281/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 70%|██████▉ | 282/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 70%|███████ | 283/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 70%|███████ | 284/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 71%|███████ | 285/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.conv1d.weight (f16, [8192,1,4]): 71%|███████ | 286/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.conv1d.bias (f16, [8192]): 71%|███████ | 287/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.A_log (f32, [32]): 71%|███████▏ | 288/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.dt_bias (f32, [32]): 72%|███████▏ | 289/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.norm.weight (f16, [128]): 72%|███████▏ | 290/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 72%|███████▏ | 291/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.post_attention_layernorm.weight (f16, [2560]): 72%|███████▏ | 292/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 73%|███████▎ | 293/404 [00:10<00:04, 26.50it/s]
Saving model.layers.22.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 73%|███████▎ | 294/404 [00:11<00:03, 27.68it/s]
Saving model.layers.22.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 73%|███████▎ | 294/404 [00:11<00:03, 27.68it/s]
Saving model.layers.23.input_layernorm.weight (f16, [2560]): 73%|███████▎ | 295/404 [00:11<00:03, 27.68it/s]
Saving model.layers.23.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 73%|███████▎ | 296/404 [00:11<00:03, 27.68it/s]
Saving model.layers.23.attn.q_norm.layernorm.weight (f16, [256]): 74%|███████▎ | 297/404 [00:11<00:03, 27.68it/s]
Saving model.layers.23.attn.k_norm.layernorm.weight (f16, [256]): 74%|███████▍ | 298/404 [00:11<00:03, 27.68it/s]
Saving model.layers.23.attn.o_proj.MatMul.weight (f16, [4096,2560]): 74%|███████▍ | 299/404 [00:11<00:03, 27.68it/s]
Saving model.layers.23.post_attention_layernorm.weight (f16, [2560]): 74%|███████▍ | 300/404 [00:11<00:03, 27.68it/s]
Saving model.layers.23.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 75%|███████▍ | 301/404 [00:11<00:03, 27.68it/s]
Saving model.layers.23.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 75%|███████▍ | 302/404 [00:11<00:04, 24.59it/s]
Saving model.layers.23.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 75%|███████▍ | 302/404 [00:11<00:04, 24.59it/s]
Saving model.layers.24.input_layernorm.weight (f16, [2560]): 75%|███████▌ | 303/404 [00:11<00:04, 24.59it/s]
Saving model.layers.24.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 75%|███████▌ | 304/404 [00:11<00:04, 24.59it/s]
Saving model.layers.24.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 75%|███████▌ | 305/404 [00:11<00:04, 24.59it/s]
Saving model.layers.24.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 76%|███████▌ | 306/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 76%|███████▌ | 307/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.linear_attn.conv1d.weight (f16, [8192,1,4]): 76%|███████▌ | 308/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.linear_attn.conv1d.bias (f16, [8192]): 76%|███████▋ | 309/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.linear_attn.A_log (f32, [32]): 77%|███████▋ | 310/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.linear_attn.dt_bias (f32, [32]): 77%|███████▋ | 311/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.linear_attn.norm.weight (f16, [128]): 77%|███████▋ | 312/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 77%|███████▋ | 313/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.post_attention_layernorm.weight (f16, [2560]): 78%|███████▊ | 314/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 78%|███████▊ | 315/404 [00:11<00:03, 24.59it/s]
Saving model.layers.24.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 78%|███████▊ | 316/404 [00:12<00:03, 25.91it/s]
Saving model.layers.24.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 78%|███████▊ | 316/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.input_layernorm.weight (f16, [2560]): 78%|███████▊ | 317/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 79%|███████▊ | 318/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 79%|███████▉ | 319/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 79%|███████▉ | 320/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 79%|███████▉ | 321/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.conv1d.weight (f16, [8192,1,4]): 80%|███████▉ | 322/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.conv1d.bias (f16, [8192]): 80%|███████▉ | 323/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.A_log (f32, [32]): 80%|████████ | 324/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.dt_bias (f32, [32]): 80%|████████ | 325/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.norm.weight (f16, [128]): 81%|████████ | 326/404 [00:12<00:03, 25.91it/s]
Saving model.layers.25.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 81%|████████ | 327/404 [00:12<00:02, 25.91it/s]
Saving model.layers.25.post_attention_layernorm.weight (f16, [2560]): 81%|████████ | 328/404 [00:12<00:02, 25.91it/s]
Saving model.layers.25.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 81%|████████▏ | 329/404 [00:12<00:02, 25.91it/s]
Saving model.layers.25.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 82%|████████▏ | 330/404 [00:12<00:02, 26.86it/s]
Saving model.layers.25.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 82%|████████▏ | 330/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.input_layernorm.weight (f16, [2560]): 82%|████████▏ | 331/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 82%|████████▏ | 332/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 82%|████████▏ | 333/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 83%|████████▎ | 334/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 83%|████████▎ | 335/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.conv1d.weight (f16, [8192,1,4]): 83%|████████▎ | 336/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.conv1d.bias (f16, [8192]): 83%|████████▎ | 337/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.A_log (f32, [32]): 84%|████████▎ | 338/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.dt_bias (f32, [32]): 84%|████████▍ | 339/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.norm.weight (f16, [128]): 84%|████████▍ | 340/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 84%|████████▍ | 341/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.post_attention_layernorm.weight (f16, [2560]): 85%|████████▍ | 342/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 85%|████████▍ | 343/404 [00:12<00:02, 26.86it/s]
Saving model.layers.26.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 85%|████████▌ | 344/404 [00:13<00:02, 27.08it/s]
Saving model.layers.26.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 85%|████████▌ | 344/404 [00:13<00:02, 27.08it/s]
Saving model.layers.27.input_layernorm.weight (f16, [2560]): 85%|████████▌ | 345/404 [00:13<00:02, 27.08it/s]
Saving model.layers.27.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 86%|████████▌ | 346/404 [00:13<00:02, 27.08it/s]
Saving model.layers.27.attn.q_norm.layernorm.weight (f16, [256]): 86%|████████▌ | 347/404 [00:13<00:02, 27.08it/s]
Saving model.layers.27.attn.k_norm.layernorm.weight (f16, [256]): 86%|████████▌ | 348/404 [00:13<00:02, 27.08it/s]
Saving model.layers.27.attn.o_proj.MatMul.weight (f16, [4096,2560]): 86%|████████▋ | 349/404 [00:13<00:02, 27.08it/s]
Saving model.layers.27.post_attention_layernorm.weight (f16, [2560]): 87%|████████▋ | 350/404 [00:13<00:01, 27.08it/s]
Saving model.layers.27.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 87%|████████▋ | 351/404 [00:13<00:01, 27.08it/s]
Saving model.layers.27.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 87%|████████▋ | 352/404 [00:13<00:02, 23.82it/s]
Saving model.layers.27.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 87%|████████▋ | 352/404 [00:13<00:02, 23.82it/s]
Saving model.layers.28.input_layernorm.weight (f16, [2560]): 87%|████████▋ | 353/404 [00:13<00:02, 23.82it/s]
Saving model.layers.28.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 88%|████████▊ | 354/404 [00:13<00:02, 23.82it/s]
Saving model.layers.28.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 88%|████████▊ | 355/404 [00:13<00:02, 23.82it/s]
Saving model.layers.28.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 88%|████████▊ | 356/404 [00:13<00:02, 23.82it/s]
Saving model.layers.28.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 88%|████████▊ | 357/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.linear_attn.conv1d.weight (f16, [8192,1,4]): 89%|████████▊ | 358/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.linear_attn.conv1d.bias (f16, [8192]): 89%|████████▉ | 359/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.linear_attn.A_log (f32, [32]): 89%|████████▉ | 360/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.linear_attn.dt_bias (f32, [32]): 89%|████████▉ | 361/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.linear_attn.norm.weight (f16, [128]): 90%|████████▉ | 362/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 90%|████████▉ | 363/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.post_attention_layernorm.weight (f16, [2560]): 90%|█████████ | 364/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 90%|█████████ | 365/404 [00:13<00:01, 23.82it/s]
Saving model.layers.28.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 91%|█████████ | 366/404 [00:14<00:01, 25.41it/s]
Saving model.layers.28.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 91%|█████████ | 366/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.input_layernorm.weight (f16, [2560]): 91%|█████████ | 367/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 91%|█████████ | 368/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 91%|█████████▏| 369/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 92%|█████████▏| 370/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 92%|█████████▏| 371/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.conv1d.weight (f16, [8192,1,4]): 92%|█████████▏| 372/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.conv1d.bias (f16, [8192]): 92%|█████████▏| 373/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.A_log (f32, [32]): 93%|█████████▎| 374/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.dt_bias (f32, [32]): 93%|█████████▎| 375/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.norm.weight (f16, [128]): 93%|█████████▎| 376/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 93%|█████████▎| 377/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.post_attention_layernorm.weight (f16, [2560]): 94%|█████████▎| 378/404 [00:14<00:01, 25.41it/s]
Saving model.layers.29.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 94%|█████████▍| 379/404 [00:14<00:00, 25.41it/s]
Saving model.layers.29.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 94%|█████████▍| 380/404 [00:14<00:00, 26.08it/s]
Saving model.layers.29.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 94%|█████████▍| 380/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.input_layernorm.weight (f16, [2560]): 94%|█████████▍| 381/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.qkv_proj.MatMul.weight (f16, [2560,8192]): 95%|█████████▍| 382/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.z_proj.MatMul.weight (f16, [2560,4096]): 95%|█████████▍| 383/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.b_proj.MatMul.weight (f16, [2560,32]): 95%|█████████▌| 384/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.a_proj.MatMul.weight (f16, [2560,32]): 95%|█████████▌| 385/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.conv1d.weight (f16, [8192,1,4]): 96%|█████████▌| 386/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.conv1d.bias (f16, [8192]): 96%|█████████▌| 387/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.A_log (f32, [32]): 96%|█████████▌| 388/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.dt_bias (f32, [32]): 96%|█████████▋| 389/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.norm.weight (f16, [128]): 97%|█████████▋| 390/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.linear_attn.out_proj.MatMul.weight (f16, [4096,2560]): 97%|█████████▋| 391/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.post_attention_layernorm.weight (f16, [2560]): 97%|█████████▋| 392/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 97%|█████████▋| 393/404 [00:14<00:00, 26.08it/s]
Saving model.layers.30.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 98%|█████████▊| 394/404 [00:15<00:00, 25.98it/s]
Saving model.layers.30.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 98%|█████████▊| 394/404 [00:15<00:00, 25.98it/s]
Saving model.layers.31.input_layernorm.weight (f16, [2560]): 98%|█████████▊| 395/404 [00:15<00:00, 25.98it/s]
Saving model.layers.31.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 98%|█████████▊| 396/404 [00:15<00:00, 25.98it/s]
Saving model.layers.31.attn.q_norm.layernorm.weight (f16, [256]): 98%|█████████▊| 397/404 [00:15<00:00, 25.98it/s]
Saving model.layers.31.attn.k_norm.layernorm.weight (f16, [256]): 99%|█████████▊| 398/404 [00:15<00:00, 25.98it/s]
Saving model.layers.31.attn.o_proj.MatMul.weight (f16, [4096,2560]): 99%|█████████▉| 399/404 [00:15<00:00, 25.98it/s]
Saving model.layers.31.post_attention_layernorm.weight (f16, [2560]): 99%|█████████▉| 400/404 [00:15<00:00, 25.98it/s]
Saving model.layers.31.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 99%|█████████▉| 401/404 [00:15<00:00, 25.98it/s]
Saving model.layers.31.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 100%|█████████▉| 402/404 [00:15<00:00, 21.55it/s]
Saving model.layers.31.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 100%|█████████▉| 402/404 [00:15<00:00, 21.55it/s]
Saving model.layers.32.final_norm_layernorm.weight (f16, [2560]): 100%|█████████▉| 403/404 [00:15<00:00, 21.55it/s]
Saving lm_head.MatMul.weight (f16, [2560,248320]): 100%|██████████| 404/404 [00:15<00:00, 21.55it/s]
Saving lm_head.MatMul.weight (f16, [2560,248320]): 100%|██████████| 404/404 [00:17<00:00, 23.52it/s]
Saving ONNX model in /home/anon/knowline/onnx/model
0it [00:00, ?it/s]
Saving model.mtp.pre_fc_norm_embedding.weight (f16, [2560]): 7%|▋ | 1/15 [00:00<00:00, 678.47it/s]
Saving model.mtp.pre_fc_norm_hidden.weight (f16, [2560]): 13%|█▎ | 2/15 [00:00<00:00, 1191.90it/s]
Saving model.mtp.fc.MatMul.weight (f16, [5120,2560]): 20%|██ | 3/15 [00:00<00:00, 1704.08it/s]
Saving model.layers.0.input_layernorm.weight (f16, [2560]): 27%|██▋ | 4/15 [00:00<00:00, 183.98it/s]
Saving model.layers.0.attn.qkv_proj.MatMul.weight (f16, [2560,10240]): 33%|███▎ | 5/15 [00:00<00:00, 227.77it/s]
Saving model.layers.0.attn.q_norm.layernorm.weight (f16, [256]): 40%|████ | 6/15 [00:00<00:00, 108.10it/s]
Saving model.layers.0.attn.k_norm.layernorm.weight (f16, [256]): 47%|████▋ | 7/15 [00:00<00:00, 120.96it/s]
Saving cos_cache (f32, [262144,32]): 53%|█████▎ | 8/15 [00:00<00:00, 137.95it/s]
Saving sin_cache (f32, [262144,32]): 60%|██████ | 9/15 [00:00<00:00, 126.02it/s]
Saving model.layers.0.attn.o_proj.MatMul.weight (f16, [4096,2560]): 67%|██████▋ | 10/15 [00:00<00:00, 125.93it/s]
Saving model.layers.0.attn.o_proj.MatMul.weight (f16, [4096,2560]): 73%|███████▎ | 11/15 [00:00<00:00, 104.02it/s]
Saving model.layers.0.post_attention_layernorm.weight (f16, [2560]): 73%|███████▎ | 11/15 [00:00<00:00, 104.02it/s]
Saving model.layers.0.mlp.gate_up_proj.MatMul.input_1 (f16, [2560,18432]): 80%|████████ | 12/15 [00:00<00:00, 104.02it/s]
Saving model.layers.0.mlp.down_proj.MatMul.weight (f16, [9216,2560]): 87%|████████▋ | 13/15 [00:00<00:00, 104.02it/s]
Saving model.layers.1.final_norm_layernorm.weight (f16, [2560]): 93%|█████████▎| 14/15 [00:00<00:00, 104.02it/s]
Saving lm_head.MatMul.weight (f16, [2560,248320]): 100%|██████████| 15/15 [00:00<00:00, 104.02it/s]
Saving lm_head.MatMul.weight (f16, [2560,248320]): 100%|██████████| 15/15 [00:01<00:00, 9.50it/s]
Configured 1271 MB of runtime initializer sharing; /home/anon/knowline/onnx/model/mtp.onnx.data remains self-contained.
Warning: could not read generation_config.json (/home/anon/LLM/KnowLine-4B-Gen5.5 does not appear to have a file named generation_config.json. Checkout 'https://huggingface.co//home/anon/LLM/KnowLine-4B-Gen5.5/tree/main' for available files.). Falling back to config.json.
Adding the tokenizer's end-of-turn token <|im_end|> (id 248046) to eos_token_id from the model configuration: 248044.
Saving GenAI config in /home/anon/knowline/onnx/model
Added 'mtp' section to genai_config.json
Saving processing files in /home/anon/knowline/onnx/model for GenAI
EXIT:0
|