| Valid precision + execution provider combinations are: FP32 CPU, FP32 CUDA, FP16 CUDA, FP16 DML, FP16 TRT-RTX, BF16 CUDA, BF16 TRT-RTX, INT8 CPU, INT8 CUDA, INT8 WebGPU, INT4 CPU, INT4 CUDA, INT4 DML, INT4 WebGPU |
| Extra options: {'accuracy_level': '4', 'block_size': '32'} |
| GroupQueryAttention (GQA) is used in this model. |
| GroupQueryAttention (GQA) is used in this model. |
|
Loading weights: 0%| | 0/426 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 426/426 [00:00<00:00, 21187.62it/s] |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[1] ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/ReduceSum ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/Sub ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/Sub/Cast ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/Shape ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/1 ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/attn_mask_reformat/attn_mask_subgraph/Gather ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - Gather doesn't have const weight. Skip quantization. |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/attn_mask_reformat/attn_mask_subgraph/Gather with 4 bits ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/Gather/Cast ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[248320, 1280] ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/embed_tokens/Reshape ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/embed_tokens/GatherBlockQuantized ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/input_layernorm/LayerNorm ... |
| 2026-11-05 03:42:47,294 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:47,348 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,348 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.0/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.0/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[2048, 2048, 4096] ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, 0, 16, 128] ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, 0, 2048] ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/FLOAT/0.08838834764831843 ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:47,370 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:47,399 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,399 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:47,399 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:47,541 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,541 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:47,541 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:47,541 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:47,541 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/mlp/Mul ... |
| 2026-11-05 03:42:47,541 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:47,586 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,586 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:47,586 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.1/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:47,632 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.1/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,632 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.1/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.1/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.1/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.1/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:47,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.1/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:47,698 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.1/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,698 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:47,698 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.1/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:47,836 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.1/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,836 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:47,836 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:47,836 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:47,836 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/mlp/Mul ... |
| 2026-11-05 03:42:47,836 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.1/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:47,882 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.1/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,882 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:47,882 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.2/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:47,931 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.2/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,931 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.2/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.2/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.2/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.2/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:47,953 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.2/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:47,974 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.2/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:47,974 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:47,974 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.2/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:48,104 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.2/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,104 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:48,104 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:48,104 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:48,104 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.2/mlp/Mul ... |
| 2026-11-05 03:42:48,104 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.2/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:48,147 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.2/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,147 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:48,147 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.3/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.3/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[8192, 1024, 1024] ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/qkv_proj/Split ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, 0, 16, 512] ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/q_gate/Reshape ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[256, 256] ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/q_gate/Split ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, 0, 4096] ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/q_proj/Reshape ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/gate/Reshape ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, -1, 256] ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, -1, 4096] ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, -1, 1024] ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/GroupQueryAttention ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/gate/Sigmoid ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/attn/gate/Mul ... |
| 2026-11-05 03:42:48,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.3/attn/o_proj/MatMul ... |
| 2026-11-05 03:42:48,213 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.3/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,213 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:48,213 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.3/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:48,349 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.3/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,349 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:48,349 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:48,349 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:48,349 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.3/mlp/Mul ... |
| 2026-11-05 03:42:48,349 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.3/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:48,398 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.3/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,398 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:48,398 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.4/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:48,443 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.4/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,443 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.4/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.4/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.4/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.4/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:48,473 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.4/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:48,494 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.4/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,494 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:48,494 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.4/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:48,642 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.4/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,642 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:48,642 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:48,642 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:48,642 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.4/mlp/Mul ... |
| 2026-11-05 03:42:48,642 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.4/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:48,686 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.4/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,686 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:48,686 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.5/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:48,733 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.5/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,733 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.5/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.5/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.5/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.5/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:48,755 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.5/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:48,777 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.5/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,777 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:48,777 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.5/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:48,935 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.5/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,935 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:48,935 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:48,935 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:48,935 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.5/mlp/Mul ... |
| 2026-11-05 03:42:48,935 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.5/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:48,997 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.5/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:48,998 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:48,998 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.6/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:49,045 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.6/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,045 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.6/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.6/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.6/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.6/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:49,068 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.6/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:49,090 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.6/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,090 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:49,090 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.6/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:49,245 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.6/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,245 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:49,245 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:49,245 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:49,245 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.6/mlp/Mul ... |
| 2026-11-05 03:42:49,245 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.6/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:49,299 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.6/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,299 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:49,299 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.7/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.7/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/qkv_proj/Split ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/q_gate/Reshape ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/q_gate/Split ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/q_proj/Reshape ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/gate/Reshape ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/GroupQueryAttention ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/gate/Sigmoid ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/attn/gate/Mul ... |
| 2026-11-05 03:42:49,379 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.7/attn/o_proj/MatMul ... |
| 2026-11-05 03:42:49,398 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.7/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,398 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:49,398 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.7/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:49,547 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.7/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,547 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:49,547 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:49,547 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:49,547 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.7/mlp/Mul ... |
| 2026-11-05 03:42:49,547 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.7/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:49,620 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.7/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,620 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:49,620 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.8/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:49,668 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.8/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,668 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.8/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:49,691 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.8/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.8/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.8/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:49,692 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.8/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:49,714 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.8/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,714 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:49,714 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.8/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:49,867 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.8/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,867 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:49,867 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:49,867 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:49,867 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.8/mlp/Mul ... |
| 2026-11-05 03:42:49,867 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.8/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:49,954 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.8/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,954 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:49,954 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.9/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:49,992 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.9/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:49,992 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.9/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:50,014 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.9/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.9/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.9/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:50,015 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.9/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:50,039 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.9/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,039 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:50,039 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.9/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:50,207 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.9/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,208 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:50,208 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:50,208 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:50,208 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.9/mlp/Mul ... |
| 2026-11-05 03:42:50,208 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.9/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:50,270 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.9/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,270 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:50,270 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.10/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:50,315 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.10/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,315 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.10/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.10/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.10/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.10/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:50,338 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.10/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:50,360 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.10/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,360 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:50,360 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.10/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:50,520 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.10/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,520 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:50,520 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:50,520 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:50,520 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.10/mlp/Mul ... |
| 2026-11-05 03:42:50,520 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.10/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:50,592 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.10/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,592 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:50,592 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.11/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.11/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/qkv_proj/Split ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/q_gate/Reshape ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/q_gate/Split ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/q_proj/Reshape ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/gate/Reshape ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/GroupQueryAttention ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/gate/Sigmoid ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/attn/gate/Mul ... |
| 2026-11-05 03:42:50,669 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.11/attn/o_proj/MatMul ... |
| 2026-11-05 03:42:50,690 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.11/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,690 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:50,690 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.11/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:50,846 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.11/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,846 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:50,846 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:50,846 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:50,846 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.11/mlp/Mul ... |
| 2026-11-05 03:42:50,846 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.11/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:50,915 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.11/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,915 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:50,915 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.12/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:50,976 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.12/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,976 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.12/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.12/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.12/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.12/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:50,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:51,000 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:51,000 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:51,000 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:51,000 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:51,000 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:51,000 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.12/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:51,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.12/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:51,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.12/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:51,199 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.12/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,199 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:51,199 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:51,199 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:51,199 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.12/mlp/Mul ... |
| 2026-11-05 03:42:51,199 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.12/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:51,262 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.12/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,262 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:51,262 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.13/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:51,314 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.13/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,314 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.13/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.13/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.13/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.13/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:51,336 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.13/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:51,363 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.13/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,364 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:51,364 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.13/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:51,536 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.13/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,536 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:51,536 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:51,536 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:51,536 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.13/mlp/Mul ... |
| 2026-11-05 03:42:51,536 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.13/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:51,614 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.13/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,614 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:51,614 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.14/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:51,664 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.14/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,664 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.14/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.14/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.14/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.14/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:51,696 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.14/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:51,731 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.14/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,731 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:51,731 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.14/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:51,885 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.14/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,885 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:51,885 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:51,885 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:51,885 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.14/mlp/Mul ... |
| 2026-11-05 03:42:51,885 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.14/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:51,947 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.14/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:51,947 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:51,947 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.15/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.15/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/qkv_proj/Split ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/q_gate/Reshape ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/q_gate/Split ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/q_proj/Reshape ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/gate/Reshape ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/GroupQueryAttention ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/gate/Sigmoid ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/attn/gate/Mul ... |
| 2026-11-05 03:42:52,009 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.15/attn/o_proj/MatMul ... |
| 2026-11-05 03:42:52,032 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.15/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,032 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:52,032 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.15/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:52,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.15/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:52,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:52,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:52,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.15/mlp/Mul ... |
| 2026-11-05 03:42:52,194 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.15/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:52,264 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.15/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,264 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:52,264 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.16/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:52,308 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.16/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,308 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.16/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.16/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.16/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.16/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:52,331 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.16/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:52,352 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.16/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,352 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:52,352 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.16/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:52,506 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.16/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,506 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:52,506 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:52,506 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:52,506 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.16/mlp/Mul ... |
| 2026-11-05 03:42:52,506 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.16/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:52,573 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.16/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,573 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:52,573 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.17/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:52,620 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.17/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,621 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.17/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:52,642 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.17/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.17/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.17/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:52,643 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.17/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:52,664 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.17/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,664 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:52,664 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.17/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:52,813 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.17/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,813 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:52,813 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:52,813 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:52,813 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.17/mlp/Mul ... |
| 2026-11-05 03:42:52,813 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.17/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:52,862 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.17/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,863 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:52,863 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.18/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:52,907 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.18/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,907 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.18/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.18/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.18/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.18/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:52,921 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.18/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:52,937 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.18/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:52,937 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:52,937 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.18/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:53,091 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.18/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,091 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:53,091 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:53,091 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:53,091 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.18/mlp/Mul ... |
| 2026-11-05 03:42:53,091 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.18/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:53,143 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.18/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,143 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:53,143 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.19/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.19/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/qkv_proj/Split ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/q_gate/Reshape ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/q_gate/Split ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/q_proj/Reshape ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/gate/Reshape ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/GroupQueryAttention ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/gate/Sigmoid ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/attn/gate/Mul ... |
| 2026-11-05 03:42:53,203 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.19/attn/o_proj/MatMul ... |
| 2026-11-05 03:42:53,227 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.19/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,227 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:53,227 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.19/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:53,371 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.19/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,371 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:53,371 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:53,372 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:53,372 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.19/mlp/Mul ... |
| 2026-11-05 03:42:53,372 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.19/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:53,458 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.19/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,459 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:53,459 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.20/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:53,503 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.20/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,503 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.20/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.20/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.20/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.20/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:53,526 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.20/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:53,561 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.20/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,561 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:53,561 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.20/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:53,704 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.20/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,704 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:53,704 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:53,704 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:53,704 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.20/mlp/Mul ... |
| 2026-11-05 03:42:53,704 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.20/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:53,754 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.20/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,754 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:53,754 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.21/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:53,808 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.21/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,808 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.21/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.21/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.21/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.21/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:53,830 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.21/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:53,853 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.21/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:53,853 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:53,853 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.21/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:54,013 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.21/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,013 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:54,013 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:54,013 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:54,013 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.21/mlp/Mul ... |
| 2026-11-05 03:42:54,013 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.21/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:54,066 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.21/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,066 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:54,066 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.22/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:54,108 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.22/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,108 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.22/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.22/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.22/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.22/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:54,131 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.22/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:54,154 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.22/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,154 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:54,154 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.22/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:54,283 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.22/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,283 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:54,283 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:54,283 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:54,283 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.22/mlp/Mul ... |
| 2026-11-05 03:42:54,283 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.22/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:54,326 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.22/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,326 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:54,326 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.23/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.23/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/qkv_proj/Split ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/q_gate/Reshape ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/q_gate/Split ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/q_proj/Reshape ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/gate/Reshape ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/GroupQueryAttention ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/gate/Sigmoid ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/attn/gate/Mul ... |
| 2026-11-05 03:42:54,383 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.23/attn/o_proj/MatMul ... |
| 2026-11-05 03:42:54,397 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.23/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,397 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:54,397 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.23/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:54,524 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.23/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,524 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:54,524 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:54,524 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:54,524 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.23/mlp/Mul ... |
| 2026-11-05 03:42:54,524 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.23/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:54,568 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.23/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,568 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:54,568 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.24/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:54,617 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.24/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,617 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.24/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:54,639 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.24/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,639 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.24/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:54,639 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.24/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:54,639 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:54,639 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:54,639 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:54,639 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:54,639 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:54,640 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.24/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:54,661 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.24/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,661 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:54,661 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.24/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:54,802 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.24/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,802 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:54,802 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:54,802 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:54,802 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.24/mlp/Mul ... |
| 2026-11-05 03:42:54,802 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.24/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:54,852 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.24/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,852 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:54,852 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.25/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:54,896 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.25/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,896 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.25/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.25/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.25/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.25/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:54,916 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.25/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:54,938 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.25/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:54,938 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:54,938 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.25/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:55,086 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.25/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,086 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:55,086 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:55,086 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:55,086 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.25/mlp/Mul ... |
| 2026-11-05 03:42:55,086 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.25/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:55,140 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.25/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,140 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:55,140 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.26/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:55,190 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.26/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,190 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.26/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.26/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.26/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.26/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:55,212 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.26/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:55,234 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.26/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,234 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:55,234 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.26/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:55,377 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.26/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,377 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:55,377 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:55,377 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:55,377 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.26/mlp/Mul ... |
| 2026-11-05 03:42:55,377 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.26/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:55,422 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.26/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,422 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:55,422 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.27/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:55,481 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.27/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/qkv_proj/Split ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/q_gate/Reshape ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/q_gate/Split ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/q_proj/Reshape ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/gate/Reshape ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/GroupQueryAttention ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/gate/Sigmoid ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/attn/gate/Mul ... |
| 2026-11-05 03:42:55,482 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.27/attn/o_proj/MatMul ... |
| 2026-11-05 03:42:55,505 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.27/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,505 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:55,505 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.27/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:55,645 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.27/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,646 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:55,646 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:55,646 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:55,646 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.27/mlp/Mul ... |
| 2026-11-05 03:42:55,646 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.27/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:55,691 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.27/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,691 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:55,691 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.28/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:55,748 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.28/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,748 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.28/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.28/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.28/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.28/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:55,775 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.28/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:55,792 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.28/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,792 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:55,792 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.28/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:55,946 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.28/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,946 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:55,946 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:55,946 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:55,946 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.28/mlp/Mul ... |
| 2026-11-05 03:42:55,946 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.28/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:55,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.28/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:55,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:55,999 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.29/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:56,048 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.29/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,048 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.29/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:56,063 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.29/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.29/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.29/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:56,064 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.29/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:56,080 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.29/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,080 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:56,080 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.29/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:56,243 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.29/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,243 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:56,243 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:56,243 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:56,243 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.29/mlp/Mul ... |
| 2026-11-05 03:42:56,243 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.29/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:56,356 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.29/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,356 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:56,357 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.30/linear_attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:56,404 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.30/linear_attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,404 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.30/linear_attn/z_proj/MatMul ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.30/linear_attn/z_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.30/linear_attn/b_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - exclude to quantize /model/layers.30/linear_attn/a_proj/MatMul as specified by nodes_to_exclude... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/qkv_proj/Transpose ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/CausalConvWithState ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/conv_out/Transpose ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/split_qkv/Split ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/q_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/q_l2norm/LpNormalization ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/q_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/k_l2norm/flat/Reshape ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/k_l2norm/LpNormalization ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/k_l2norm/unflat/Reshape ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/q_scaled/Mul ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/LinearAttentionGate ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/LinearAttention ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/linear_attn/GatedRMSNorm ... |
| 2026-11-05 03:42:56,426 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.30/linear_attn/out_proj/MatMul ... |
| 2026-11-05 03:42:56,449 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.30/linear_attn/out_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,449 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:56,449 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.30/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:56,602 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.30/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,602 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:56,602 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:56,602 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:56,602 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.30/mlp/Mul ... |
| 2026-11-05 03:42:56,602 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.30/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:56,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.30/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/input_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:56,654 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.31/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:42:56,717 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.31/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,717 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/qkv_proj/Split ... |
| 2026-11-05 03:42:56,717 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/q_gate/Reshape ... |
| 2026-11-05 03:42:56,717 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/q_gate/Split ... |
| 2026-11-05 03:42:56,717 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/q_proj/Reshape ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/gate/Reshape ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/GroupQueryAttention ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/gate/Sigmoid ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/attn/gate/Mul ... |
| 2026-11-05 03:42:56,718 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.31/attn/o_proj/MatMul ... |
| 2026-11-05 03:42:56,731 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.31/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,731 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:56,731 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.31/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:42:56,856 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.31/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,856 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:42:56,856 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:42:56,856 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/mlp/act_fn/Mul ... |
| 2026-11-05 03:42:56,856 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.31/mlp/Mul ... |
| 2026-11-05 03:42:56,856 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.31/mlp/down_proj/MatMul ... |
| 2026-11-05 03:42:56,906 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.31/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:42:56,906 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.32/final_norm_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:42:56,906 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /lm_head/MatMul ... |
| 2026-11-05 03:42:59,292 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /lm_head/MatMul with 4 bits ... |
| Reading embedding layer |
| Reading layer 0 |
| Reading layer 1 |
| Reading layer 2 |
| Reading layer 3 |
| Reading layer 4 |
| Reading layer 5 |
| Reading layer 6 |
| Reading layer 7 |
| Reading layer 8 |
| Reading layer 9 |
| Reading layer 10 |
| Reading layer 11 |
| Reading layer 12 |
| Reading layer 13 |
| Reading layer 14 |
| Reading layer 15 |
| Reading layer 16 |
| Reading layer 17 |
| Reading layer 18 |
| Reading layer 19 |
| Reading layer 20 |
| Reading layer 21 |
| Reading layer 22 |
| Reading layer 23 |
| Reading layer 24 |
| Reading layer 25 |
| Reading layer 26 |
| Reading layer 27 |
| Reading layer 28 |
| Reading layer 29 |
| Reading layer 30 |
| Reading layer 31 |
| Reading final norm |
| Reading LM head |
| Building MTP (multi-token prediction) head -> mtp.onnx |
| Saving ONNX model in /home/anon/knowline/onnx/model-cpu |
|
0it [00:00, ?it/s]
Saving model.layers.0.input_layernorm.weight (f32, [2560]): 0%| | 1/557 [00:00<00:04, 132.46it/s]
Saving model.layers.0.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 0%| | 2/557 [00:00<00:02, 261.56it/s]
Saving model.layers.0.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 1%| | 3/557 [00:00<00:01, 383.04it/s]
Saving model.layers.0.linear_attn.conv1d.weight (f32, [8192,1,4]): 1%| | 4/557 [00:00<00:01, 504.65it/s]
Saving model.layers.0.linear_attn.conv1d.bias (f32, [8192]): 1%| | 5/557 [00:00<00:00, 625.77it/s]
Saving model.layers.0.linear_attn.dt_bias (f32, [32]): 1%| | 6/557 [00:00<00:00, 747.03it/s]
Saving model.layers.0.linear_attn.neg_exp_A (f32, [32]): 1%|▏ | 7/557 [00:00<00:00, 869.00it/s]
Saving model.layers.0.linear_attn.norm.weight (f32, [128]): 1%|▏ | 8/557 [00:00<00:00, 989.69it/s]
Saving model.layers.0.post_attention_layernorm.weight (f32, [2560]): 2%|▏ | 9/557 [00:00<00:00, 1109.93it/s]
Saving model.layers.1.input_layernorm.weight (f32, [2560]): 2%|▏ | 10/557 [00:00<00:00, 1228.06it/s]
Saving model.layers.1.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 2%|▏ | 11/557 [00:00<00:00, 1345.58it/s]
Saving model.layers.1.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 2%|▏ | 12/557 [00:00<00:00, 1453.37it/s]
Saving model.layers.1.linear_attn.conv1d.weight (f32, [8192,1,4]): 2%|▏ | 13/557 [00:00<00:00, 1559.04it/s]
Saving model.layers.1.linear_attn.conv1d.bias (f32, [8192]): 3%|▎ | 14/557 [00:00<00:00, 1657.92it/s]
Saving model.layers.1.linear_attn.dt_bias (f32, [32]): 3%|▎ | 15/557 [00:00<00:00, 1767.46it/s]
Saving model.layers.1.linear_attn.neg_exp_A (f32, [32]): 3%|▎ | 16/557 [00:00<00:00, 1880.27it/s]
Saving model.layers.1.linear_attn.norm.weight (f32, [128]): 3%|▎ | 17/557 [00:00<00:00, 1991.93it/s]
Saving model.layers.1.post_attention_layernorm.weight (f32, [2560]): 3%|▎ | 18/557 [00:00<00:00, 2103.29it/s]
Saving model.layers.2.input_layernorm.weight (f32, [2560]): 3%|▎ | 19/557 [00:00<00:00, 2212.31it/s]
Saving model.layers.2.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 4%|▎ | 20/557 [00:00<00:00, 2321.14it/s]
Saving model.layers.2.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 4%|▍ | 21/557 [00:00<00:00, 2415.48it/s]
Saving model.layers.2.linear_attn.conv1d.weight (f32, [8192,1,4]): 4%|▍ | 22/557 [00:00<00:00, 2505.56it/s]
Saving model.layers.2.linear_attn.conv1d.bias (f32, [8192]): 4%|▍ | 23/557 [00:00<00:00, 2603.61it/s]
Saving model.layers.2.linear_attn.dt_bias (f32, [32]): 4%|▍ | 24/557 [00:00<00:00, 2705.78it/s]
Saving model.layers.2.linear_attn.neg_exp_A (f32, [32]): 4%|▍ | 25/557 [00:00<00:00, 2811.35it/s]
Saving model.layers.2.linear_attn.norm.weight (f32, [128]): 5%|▍ | 26/557 [00:00<00:00, 2915.83it/s]
Saving model.layers.2.post_attention_layernorm.weight (f32, [2560]): 5%|▍ | 27/557 [00:00<00:00, 3020.22it/s]
Saving model.layers.3.input_layernorm.weight (f32, [2560]): 5%|▌ | 28/557 [00:00<00:00, 3120.68it/s]
Saving model.layers.3.attn.q_norm.layernorm.weight (f32, [256]): 5%|▌ | 29/557 [00:00<00:00, 3221.18it/s]
Saving model.layers.3.attn.k_norm.layernorm.weight (f32, [256]): 5%|▌ | 30/557 [00:00<00:00, 3323.80it/s]
Saving cos_cache (f32, [262144,32]): 6%|▌ | 31/557 [00:00<00:00, 3425.46it/s]
Saving sin_cache (f32, [262144,32]): 6%|▌ | 32/557 [00:00<00:00, 670.12it/s]
Saving model.layers.3.post_attention_layernorm.weight (f32, [2560]): 6%|▌ | 33/557 [00:00<00:01, 475.07it/s]
Saving model.layers.4.input_layernorm.weight (f32, [2560]): 6%|▌ | 34/557 [00:00<00:01, 488.71it/s]
Saving model.layers.4.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 6%|▋ | 35/557 [00:00<00:01, 502.78it/s]
Saving model.layers.4.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 6%|▋ | 36/557 [00:00<00:01, 516.44it/s]
Saving model.layers.4.linear_attn.conv1d.weight (f32, [8192,1,4]): 7%|▋ | 37/557 [00:00<00:00, 530.16it/s]
Saving model.layers.4.linear_attn.conv1d.bias (f32, [8192]): 7%|▋ | 38/557 [00:00<00:00, 544.05it/s]
Saving model.layers.4.linear_attn.dt_bias (f32, [32]): 7%|▋ | 39/557 [00:00<00:00, 558.07it/s]
Saving model.layers.4.linear_attn.neg_exp_A (f32, [32]): 7%|▋ | 40/557 [00:00<00:00, 572.20it/s]
Saving model.layers.4.linear_attn.norm.weight (f32, [128]): 7%|▋ | 41/557 [00:00<00:00, 586.30it/s]
Saving model.layers.4.post_attention_layernorm.weight (f32, [2560]): 8%|▊ | 42/557 [00:00<00:00, 600.41it/s]
Saving model.layers.5.input_layernorm.weight (f32, [2560]): 8%|▊ | 43/557 [00:00<00:00, 614.43it/s]
Saving model.layers.5.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 8%|▊ | 44/557 [00:00<00:00, 628.44it/s]
Saving model.layers.5.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 8%|▊ | 45/557 [00:00<00:00, 642.04it/s]
Saving model.layers.5.linear_attn.conv1d.weight (f32, [8192,1,4]): 8%|▊ | 46/557 [00:00<00:00, 655.52it/s]
Saving model.layers.5.linear_attn.conv1d.bias (f32, [8192]): 8%|▊ | 47/557 [00:00<00:00, 669.16it/s]
Saving model.layers.5.linear_attn.dt_bias (f32, [32]): 9%|▊ | 48/557 [00:00<00:00, 683.01it/s]
Saving model.layers.5.linear_attn.neg_exp_A (f32, [32]): 9%|▉ | 49/557 [00:00<00:00, 697.02it/s]
Saving model.layers.5.linear_attn.norm.weight (f32, [128]): 9%|▉ | 50/557 [00:00<00:00, 711.01it/s]
Saving model.layers.5.post_attention_layernorm.weight (f32, [2560]): 9%|▉ | 51/557 [00:00<00:00, 725.01it/s]
Saving model.layers.6.input_layernorm.weight (f32, [2560]): 9%|▉ | 52/557 [00:00<00:00, 738.84it/s]
Saving model.layers.6.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 10%|▉ | 53/557 [00:00<00:00, 752.74it/s]
Saving model.layers.6.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 10%|▉ | 54/557 [00:00<00:00, 766.19it/s]
Saving model.layers.6.linear_attn.conv1d.weight (f32, [8192,1,4]): 10%|▉ | 55/557 [00:00<00:00, 779.53it/s]
Saving model.layers.6.linear_attn.conv1d.bias (f32, [8192]): 10%|█ | 56/557 [00:00<00:00, 793.07it/s]
Saving model.layers.6.linear_attn.dt_bias (f32, [32]): 10%|█ | 57/557 [00:00<00:00, 806.80it/s]
Saving model.layers.6.linear_attn.neg_exp_A (f32, [32]): 10%|█ | 58/557 [00:00<00:00, 820.70it/s]
Saving model.layers.6.linear_attn.norm.weight (f32, [128]): 11%|█ | 59/557 [00:00<00:00, 834.57it/s]
Saving model.layers.6.post_attention_layernorm.weight (f32, [2560]): 11%|█ | 60/557 [00:00<00:00, 848.45it/s]
Saving model.layers.7.input_layernorm.weight (f32, [2560]): 11%|█ | 61/557 [00:00<00:00, 862.23it/s]
Saving model.layers.7.attn.q_norm.layernorm.weight (f32, [256]): 11%|█ | 62/557 [00:00<00:00, 876.00it/s]
Saving model.layers.7.attn.k_norm.layernorm.weight (f32, [256]): 11%|█▏ | 63/557 [00:00<00:00, 889.88it/s]
Saving model.layers.7.post_attention_layernorm.weight (f32, [2560]): 11%|█▏ | 64/557 [00:00<00:00, 903.68it/s]
Saving model.layers.8.input_layernorm.weight (f32, [2560]): 12%|█▏ | 65/557 [00:00<00:00, 917.41it/s]
Saving model.layers.8.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 12%|█▏ | 66/557 [00:00<00:00, 931.16it/s]
Saving model.layers.8.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 12%|█▏ | 67/557 [00:00<00:00, 944.39it/s]
Saving model.layers.8.linear_attn.conv1d.weight (f32, [8192,1,4]): 12%|█▏ | 68/557 [00:00<00:00, 957.51it/s]
Saving model.layers.8.linear_attn.conv1d.bias (f32, [8192]): 12%|█▏ | 69/557 [00:00<00:00, 970.94it/s]
Saving model.layers.8.linear_attn.dt_bias (f32, [32]): 13%|█▎ | 70/557 [00:00<00:00, 984.49it/s]
Saving model.layers.8.linear_attn.neg_exp_A (f32, [32]): 13%|█▎ | 71/557 [00:00<00:00, 998.27it/s]
Saving model.layers.8.linear_attn.norm.weight (f32, [128]): 13%|█▎ | 72/557 [00:00<00:00, 1012.02it/s]
Saving model.layers.8.post_attention_layernorm.weight (f32, [2560]): 13%|█▎ | 73/557 [00:00<00:00, 1025.70it/s]
Saving model.layers.9.input_layernorm.weight (f32, [2560]): 13%|█▎ | 74/557 [00:00<00:00, 1039.32it/s]
Saving model.layers.9.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 13%|█▎ | 75/557 [00:00<00:00, 1052.97it/s]
Saving model.layers.9.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 14%|█▎ | 76/557 [00:00<00:00, 1065.73it/s]
Saving model.layers.9.linear_attn.conv1d.weight (f32, [8192,1,4]): 14%|█▍ | 77/557 [00:00<00:00, 1078.41it/s]
Saving model.layers.9.linear_attn.conv1d.bias (f32, [8192]): 14%|█▍ | 78/557 [00:00<00:00, 1091.30it/s]
Saving model.layers.9.linear_attn.dt_bias (f32, [32]): 14%|█▍ | 79/557 [00:00<00:00, 1104.52it/s]
Saving model.layers.9.linear_attn.neg_exp_A (f32, [32]): 14%|█▍ | 80/557 [00:00<00:00, 1118.03it/s]
Saving model.layers.9.linear_attn.norm.weight (f32, [128]): 15%|█▍ | 81/557 [00:00<00:00, 1131.50it/s]
Saving model.layers.9.post_attention_layernorm.weight (f32, [2560]): 15%|█▍ | 82/557 [00:00<00:00, 1144.97it/s]
Saving model.layers.10.input_layernorm.weight (f32, [2560]): 15%|█▍ | 83/557 [00:00<00:00, 1158.29it/s]
Saving model.layers.10.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 15%|█▌ | 84/557 [00:00<00:00, 1171.60it/s]
Saving model.layers.10.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 15%|█▌ | 85/557 [00:00<00:00, 1184.09it/s]
Saving model.layers.10.linear_attn.conv1d.weight (f32, [8192,1,4]): 15%|█▌ | 86/557 [00:00<00:00, 1196.34it/s]
Saving model.layers.10.linear_attn.conv1d.bias (f32, [8192]): 16%|█▌ | 87/557 [00:00<00:00, 1209.03it/s]
Saving model.layers.10.linear_attn.dt_bias (f32, [32]): 16%|█▌ | 88/557 [00:00<00:00, 1222.12it/s]
Saving model.layers.10.linear_attn.neg_exp_A (f32, [32]): 16%|█▌ | 89/557 [00:00<00:00, 1235.49it/s]
Saving model.layers.10.linear_attn.norm.weight (f32, [128]): 16%|█▌ | 90/557 [00:00<00:00, 1248.81it/s]
Saving model.layers.10.post_attention_layernorm.weight (f32, [2560]): 16%|█▋ | 91/557 [00:00<00:00, 1262.06it/s]
Saving model.layers.11.input_layernorm.weight (f32, [2560]): 17%|█▋ | 92/557 [00:00<00:00, 1275.20it/s]
Saving model.layers.11.attn.q_norm.layernorm.weight (f32, [256]): 17%|█▋ | 93/557 [00:00<00:00, 1288.27it/s]
Saving model.layers.11.attn.k_norm.layernorm.weight (f32, [256]): 17%|█▋ | 94/557 [00:00<00:00, 1301.56it/s]
Saving model.layers.11.post_attention_layernorm.weight (f32, [2560]): 17%|█▋ | 95/557 [00:00<00:00, 1314.81it/s]
Saving model.layers.12.input_layernorm.weight (f32, [2560]): 17%|█▋ | 96/557 [00:00<00:00, 1327.88it/s]
Saving model.layers.12.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 17%|█▋ | 97/557 [00:00<00:00, 1340.93it/s]
Saving model.layers.12.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 18%|█▊ | 98/557 [00:00<00:00, 1352.34it/s]
Saving model.layers.12.linear_attn.conv1d.weight (f32, [8192,1,4]): 18%|█▊ | 99/557 [00:00<00:00, 1364.33it/s]
Saving model.layers.12.linear_attn.conv1d.bias (f32, [8192]): 18%|█▊ | 100/557 [00:00<00:00, 1377.01it/s]
Saving model.layers.12.linear_attn.dt_bias (f32, [32]): 18%|█▊ | 101/557 [00:00<00:00, 1390.05it/s]
Saving model.layers.12.linear_attn.neg_exp_A (f32, [32]): 18%|█▊ | 102/557 [00:00<00:00, 1403.40it/s]
Saving model.layers.12.linear_attn.norm.weight (f32, [128]): 18%|█▊ | 103/557 [00:00<00:00, 1416.71it/s]
Saving model.layers.12.post_attention_layernorm.weight (f32, [2560]): 19%|█▊ | 104/557 [00:00<00:00, 1430.03it/s]
Saving model.layers.13.input_layernorm.weight (f32, [2560]): 19%|█▉ | 105/557 [00:00<00:00, 1443.19it/s]
Saving model.layers.13.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 19%|█▉ | 106/557 [00:00<00:00, 1456.39it/s]
Saving model.layers.13.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 19%|█▉ | 107/557 [00:00<00:00, 1468.57it/s]
Saving model.layers.13.linear_attn.conv1d.weight (f32, [8192,1,4]): 19%|█▉ | 108/557 [00:00<00:00, 1480.73it/s]
Saving model.layers.13.linear_attn.conv1d.bias (f32, [8192]): 20%|█▉ | 109/557 [00:00<00:00, 1493.21it/s]
Saving model.layers.13.linear_attn.dt_bias (f32, [32]): 20%|█▉ | 110/557 [00:00<00:00, 1505.81it/s]
Saving model.layers.13.linear_attn.neg_exp_A (f32, [32]): 20%|█▉ | 111/557 [00:00<00:00, 1518.83it/s]
Saving model.layers.13.linear_attn.norm.weight (f32, [128]): 20%|██ | 112/557 [00:00<00:00, 1531.73it/s]
Saving model.layers.13.post_attention_layernorm.weight (f32, [2560]): 20%|██ | 113/557 [00:00<00:00, 1544.59it/s]
Saving model.layers.14.input_layernorm.weight (f32, [2560]): 20%|██ | 114/557 [00:00<00:00, 1557.19it/s]
Saving model.layers.14.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 21%|██ | 115/557 [00:00<00:00, 1569.90it/s]
Saving model.layers.14.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 21%|██ | 116/557 [00:00<00:00, 1581.42it/s]
Saving model.layers.14.linear_attn.conv1d.weight (f32, [8192,1,4]): 21%|██ | 117/557 [00:00<00:00, 1592.90it/s]
Saving model.layers.14.linear_attn.conv1d.bias (f32, [8192]): 21%|██ | 118/557 [00:00<00:00, 1604.94it/s]
Saving model.layers.14.linear_attn.dt_bias (f32, [32]): 21%|██▏ | 119/557 [00:00<00:00, 1617.37it/s]
Saving model.layers.14.linear_attn.neg_exp_A (f32, [32]): 22%|██▏ | 120/557 [00:00<00:00, 1630.37it/s]
Saving model.layers.14.linear_attn.norm.weight (f32, [128]): 22%|██▏ | 121/557 [00:00<00:00, 1643.34it/s]
Saving model.layers.14.post_attention_layernorm.weight (f32, [2560]): 22%|██▏ | 122/557 [00:00<00:00, 1656.37it/s]
Saving model.layers.15.input_layernorm.weight (f32, [2560]): 22%|██▏ | 123/557 [00:00<00:00, 1669.25it/s]
Saving model.layers.15.attn.q_norm.layernorm.weight (f32, [256]): 22%|██▏ | 124/557 [00:00<00:00, 1682.13it/s]
Saving model.layers.15.attn.k_norm.layernorm.weight (f32, [256]): 22%|██▏ | 125/557 [00:00<00:00, 1695.19it/s]
Saving model.layers.15.post_attention_layernorm.weight (f32, [2560]): 23%|██▎ | 126/557 [00:00<00:00, 1708.17it/s]
Saving model.layers.16.input_layernorm.weight (f32, [2560]): 23%|██▎ | 127/557 [00:00<00:00, 1720.97it/s]
Saving model.layers.16.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 23%|██▎ | 128/557 [00:00<00:00, 1733.82it/s]
Saving model.layers.16.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 23%|██▎ | 129/557 [00:00<00:00, 1745.69it/s]
Saving model.layers.16.linear_attn.conv1d.weight (f32, [8192,1,4]): 23%|██▎ | 130/557 [00:00<00:00, 1757.21it/s]
Saving model.layers.16.linear_attn.conv1d.bias (f32, [8192]): 24%|██▎ | 131/557 [00:00<00:00, 1769.17it/s]
Saving model.layers.16.linear_attn.dt_bias (f32, [32]): 24%|██▎ | 132/557 [00:00<00:00, 1781.70it/s]
Saving model.layers.16.linear_attn.neg_exp_A (f32, [32]): 24%|██▍ | 133/557 [00:00<00:00, 1794.68it/s]
Saving model.layers.16.linear_attn.norm.weight (f32, [128]): 24%|██▍ | 134/557 [00:00<00:00, 1807.56it/s]
Saving model.layers.16.post_attention_layernorm.weight (f32, [2560]): 24%|██▍ | 135/557 [00:00<00:00, 1820.40it/s]
Saving model.layers.17.input_layernorm.weight (f32, [2560]): 24%|██▍ | 136/557 [00:00<00:00, 1833.04it/s]
Saving model.layers.17.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 25%|██▍ | 137/557 [00:00<00:00, 1845.68it/s]
Saving model.layers.17.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 25%|██▍ | 138/557 [00:00<00:00, 1857.04it/s]
Saving model.layers.17.linear_attn.conv1d.weight (f32, [8192,1,4]): 25%|██▍ | 139/557 [00:00<00:00, 1868.13it/s]
Saving model.layers.17.linear_attn.conv1d.bias (f32, [8192]): 25%|██▌ | 140/557 [00:00<00:00, 1879.82it/s]
Saving model.layers.17.linear_attn.dt_bias (f32, [32]): 25%|██▌ | 141/557 [00:00<00:00, 1892.03it/s]
Saving model.layers.17.linear_attn.neg_exp_A (f32, [32]): 25%|██▌ | 142/557 [00:00<00:00, 1904.89it/s]
Saving model.layers.17.linear_attn.norm.weight (f32, [128]): 26%|██▌ | 143/557 [00:00<00:00, 1917.63it/s]
Saving model.layers.17.post_attention_layernorm.weight (f32, [2560]): 26%|██▌ | 144/557 [00:00<00:00, 1930.43it/s]
Saving model.layers.18.input_layernorm.weight (f32, [2560]): 26%|██▌ | 145/557 [00:00<00:00, 1942.88it/s]
Saving model.layers.18.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 26%|██▌ | 146/557 [00:00<00:00, 1955.35it/s]
Saving model.layers.18.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 26%|██▋ | 147/557 [00:00<00:00, 1966.49it/s]
Saving model.layers.18.linear_attn.conv1d.weight (f32, [8192,1,4]): 27%|██▋ | 148/557 [00:00<00:00, 1977.82it/s]
Saving model.layers.18.linear_attn.conv1d.bias (f32, [8192]): 27%|██▋ | 149/557 [00:00<00:00, 1989.79it/s]
Saving model.layers.18.linear_attn.dt_bias (f32, [32]): 27%|██▋ | 150/557 [00:00<00:00, 2002.11it/s]
Saving model.layers.18.linear_attn.neg_exp_A (f32, [32]): 27%|██▋ | 151/557 [00:00<00:00, 2014.86it/s]
Saving model.layers.18.linear_attn.norm.weight (f32, [128]): 27%|██▋ | 152/557 [00:00<00:00, 2027.51it/s]
Saving model.layers.18.post_attention_layernorm.weight (f32, [2560]): 27%|██▋ | 153/557 [00:00<00:00, 2040.21it/s]
Saving model.layers.19.input_layernorm.weight (f32, [2560]): 28%|██▊ | 154/557 [00:00<00:00, 2052.67it/s]
Saving model.layers.19.attn.q_norm.layernorm.weight (f32, [256]): 28%|██▊ | 155/557 [00:00<00:00, 2065.16it/s]
Saving model.layers.19.attn.k_norm.layernorm.weight (f32, [256]): 28%|██▊ | 156/557 [00:00<00:00, 2077.88it/s]
Saving model.layers.19.post_attention_layernorm.weight (f32, [2560]): 28%|██▊ | 157/557 [00:00<00:00, 2090.51it/s]
Saving model.layers.20.input_layernorm.weight (f32, [2560]): 28%|██▊ | 158/557 [00:00<00:00, 2102.93it/s]
Saving model.layers.20.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 29%|██▊ | 159/557 [00:00<00:00, 2115.26it/s]
Saving model.layers.20.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 29%|██▊ | 160/557 [00:00<00:00, 2126.32it/s]
Saving model.layers.20.linear_attn.conv1d.weight (f32, [8192,1,4]): 29%|██▉ | 161/557 [00:00<00:00, 2137.38it/s]
Saving model.layers.20.linear_attn.conv1d.bias (f32, [8192]): 29%|██▉ | 162/557 [00:00<00:00, 2148.63it/s]
Saving model.layers.20.linear_attn.dt_bias (f32, [32]): 29%|██▉ | 163/557 [00:00<00:00, 2160.67it/s]
Saving model.layers.20.linear_attn.neg_exp_A (f32, [32]): 29%|██▉ | 164/557 [00:00<00:00, 2173.23it/s]
Saving model.layers.20.linear_attn.norm.weight (f32, [128]): 30%|██▉ | 165/557 [00:00<00:00, 2185.76it/s]
Saving model.layers.20.post_attention_layernorm.weight (f32, [2560]): 30%|██▉ | 166/557 [00:00<00:00, 2198.29it/s]
Saving model.layers.21.input_layernorm.weight (f32, [2560]): 30%|██▉ | 167/557 [00:00<00:00, 2210.53it/s]
Saving model.layers.21.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 30%|███ | 168/557 [00:00<00:00, 2222.87it/s]
Saving model.layers.21.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 30%|███ | 169/557 [00:00<00:00, 2233.83it/s]
Saving model.layers.21.linear_attn.conv1d.weight (f32, [8192,1,4]): 31%|███ | 170/557 [00:00<00:00, 2244.45it/s]
Saving model.layers.21.linear_attn.conv1d.bias (f32, [8192]): 31%|███ | 171/557 [00:00<00:00, 2256.04it/s]
Saving model.layers.21.linear_attn.dt_bias (f32, [32]): 31%|███ | 172/557 [00:00<00:00, 2268.09it/s]
Saving model.layers.21.linear_attn.neg_exp_A (f32, [32]): 31%|███ | 173/557 [00:00<00:00, 2280.62it/s]
Saving model.layers.21.linear_attn.norm.weight (f32, [128]): 31%|███ | 174/557 [00:00<00:00, 2293.05it/s]
Saving model.layers.21.post_attention_layernorm.weight (f32, [2560]): 31%|███▏ | 175/557 [00:00<00:00, 2301.72it/s]
Saving model.layers.22.input_layernorm.weight (f32, [2560]): 32%|███▏ | 176/557 [00:00<00:00, 2313.40it/s]
Saving model.layers.22.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 32%|███▏ | 177/557 [00:00<00:00, 2325.49it/s]
Saving model.layers.22.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 32%|███▏ | 178/557 [00:00<00:00, 2335.88it/s]
Saving model.layers.22.linear_attn.conv1d.weight (f32, [8192,1,4]): 32%|███▏ | 179/557 [00:00<00:00, 2346.29it/s]
Saving model.layers.22.linear_attn.conv1d.bias (f32, [8192]): 32%|███▏ | 180/557 [00:00<00:00, 2357.51it/s]
Saving model.layers.22.linear_attn.dt_bias (f32, [32]): 32%|███▏ | 181/557 [00:00<00:00, 2369.26it/s]
Saving model.layers.22.linear_attn.neg_exp_A (f32, [32]): 33%|███▎ | 182/557 [00:00<00:00, 2381.65it/s]
Saving model.layers.22.linear_attn.norm.weight (f32, [128]): 33%|███▎ | 183/557 [00:00<00:00, 2393.97it/s]
Saving model.layers.22.post_attention_layernorm.weight (f32, [2560]): 33%|███▎ | 184/557 [00:00<00:00, 2406.35it/s]
Saving model.layers.23.input_layernorm.weight (f32, [2560]): 33%|███▎ | 185/557 [00:00<00:00, 2418.38it/s]
Saving model.layers.23.attn.q_norm.layernorm.weight (f32, [256]): 33%|███▎ | 186/557 [00:00<00:00, 2430.47it/s]
Saving model.layers.23.attn.k_norm.layernorm.weight (f32, [256]): 34%|███▎ | 187/557 [00:00<00:00, 2442.86it/s]
Saving model.layers.23.post_attention_layernorm.weight (f32, [2560]): 34%|███▍ | 188/557 [00:00<00:00, 2455.15it/s]
Saving model.layers.24.input_layernorm.weight (f32, [2560]): 34%|███▍ | 189/557 [00:00<00:00, 2467.12it/s]
Saving model.layers.24.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 34%|███▍ | 190/557 [00:00<00:00, 2479.12it/s]
Saving model.layers.24.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 34%|███▍ | 191/557 [00:00<00:00, 2489.53it/s]
Saving model.layers.24.linear_attn.conv1d.weight (f32, [8192,1,4]): 34%|███▍ | 192/557 [00:00<00:00, 2500.07it/s]
Saving model.layers.24.linear_attn.conv1d.bias (f32, [8192]): 35%|███▍ | 193/557 [00:00<00:00, 2511.22it/s]
Saving model.layers.24.linear_attn.dt_bias (f32, [32]): 35%|███▍ | 194/557 [00:00<00:00, 2522.98it/s]
Saving model.layers.24.linear_attn.neg_exp_A (f32, [32]): 35%|███▌ | 195/557 [00:00<00:00, 2535.23it/s]
Saving model.layers.24.linear_attn.norm.weight (f32, [128]): 35%|███▌ | 196/557 [00:00<00:00, 2547.38it/s]
Saving model.layers.24.post_attention_layernorm.weight (f32, [2560]): 35%|███▌ | 197/557 [00:00<00:00, 2559.63it/s]
Saving model.layers.25.input_layernorm.weight (f32, [2560]): 36%|███▌ | 198/557 [00:00<00:00, 2571.47it/s]
Saving model.layers.25.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 36%|███▌ | 199/557 [00:00<00:00, 2583.46it/s]
Saving model.layers.25.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 36%|███▌ | 200/557 [00:00<00:00, 2593.50it/s]
Saving model.layers.25.linear_attn.conv1d.weight (f32, [8192,1,4]): 36%|███▌ | 201/557 [00:00<00:00, 2603.60it/s]
Saving model.layers.25.linear_attn.conv1d.bias (f32, [8192]): 36%|███▋ | 202/557 [00:00<00:00, 2614.68it/s]
Saving model.layers.25.linear_attn.dt_bias (f32, [32]): 36%|███▋ | 203/557 [00:00<00:00, 2626.40it/s]
Saving model.layers.25.linear_attn.neg_exp_A (f32, [32]): 37%|███▋ | 204/557 [00:00<00:00, 2638.63it/s]
Saving model.layers.25.linear_attn.norm.weight (f32, [128]): 37%|███▋ | 205/557 [00:00<00:00, 2650.77it/s]
Saving model.layers.25.post_attention_layernorm.weight (f32, [2560]): 37%|███▋ | 206/557 [00:00<00:00, 2662.87it/s]
Saving model.layers.26.input_layernorm.weight (f32, [2560]): 37%|███▋ | 207/557 [00:00<00:00, 2674.42it/s]
Saving model.layers.26.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 37%|███▋ | 208/557 [00:00<00:00, 2686.23it/s]
Saving model.layers.26.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 38%|███▊ | 209/557 [00:00<00:00, 2696.66it/s]
Saving model.layers.26.linear_attn.conv1d.weight (f32, [8192,1,4]): 38%|███▊ | 210/557 [00:00<00:00, 2707.02it/s]
Saving model.layers.26.linear_attn.conv1d.bias (f32, [8192]): 38%|███▊ | 211/557 [00:00<00:00, 2718.25it/s]
Saving model.layers.26.linear_attn.dt_bias (f32, [32]): 38%|███▊ | 212/557 [00:00<00:00, 2729.90it/s]
Saving model.layers.26.linear_attn.neg_exp_A (f32, [32]): 38%|███▊ | 213/557 [00:00<00:00, 2742.07it/s]
Saving model.layers.26.linear_attn.norm.weight (f32, [128]): 38%|███▊ | 214/557 [00:00<00:00, 2754.09it/s]
Saving model.layers.26.post_attention_layernorm.weight (f32, [2560]): 39%|███▊ | 215/557 [00:00<00:00, 2766.17it/s]
Saving model.layers.27.input_layernorm.weight (f32, [2560]): 39%|███▉ | 216/557 [00:00<00:00, 2778.00it/s]
Saving model.layers.27.attn.q_norm.layernorm.weight (f32, [256]): 39%|███▉ | 217/557 [00:00<00:00, 2789.83it/s]
Saving model.layers.27.attn.k_norm.layernorm.weight (f32, [256]): 39%|███▉ | 218/557 [00:00<00:00, 2801.96it/s]
Saving model.layers.27.post_attention_layernorm.weight (f32, [2560]): 39%|███▉ | 219/557 [00:00<00:00, 2813.96it/s]
Saving model.layers.28.input_layernorm.weight (f32, [2560]): 39%|███▉ | 220/557 [00:00<00:00, 2825.72it/s]
Saving model.layers.28.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 40%|███▉ | 221/557 [00:00<00:00, 2837.40it/s]
Saving model.layers.28.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 40%|███▉ | 222/557 [00:00<00:00, 2847.75it/s]
Saving model.layers.28.linear_attn.conv1d.weight (f32, [8192,1,4]): 40%|████ | 223/557 [00:00<00:00, 2857.76it/s]
Saving model.layers.28.linear_attn.conv1d.bias (f32, [8192]): 40%|████ | 224/557 [00:00<00:00, 2868.74it/s]
Saving model.layers.28.linear_attn.dt_bias (f32, [32]): 40%|████ | 225/557 [00:00<00:00, 2880.25it/s]
Saving model.layers.28.linear_attn.neg_exp_A (f32, [32]): 41%|████ | 226/557 [00:00<00:00, 2892.27it/s]
Saving model.layers.28.linear_attn.norm.weight (f32, [128]): 41%|████ | 227/557 [00:00<00:00, 2904.14it/s]
Saving model.layers.28.post_attention_layernorm.weight (f32, [2560]): 41%|████ | 228/557 [00:00<00:00, 2915.92it/s]
Saving model.layers.29.input_layernorm.weight (f32, [2560]): 41%|████ | 229/557 [00:00<00:00, 2927.52it/s]
Saving model.layers.29.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 41%|████▏ | 230/557 [00:00<00:00, 2939.25it/s]
Saving model.layers.29.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 41%|████▏ | 231/557 [00:00<00:00, 2949.31it/s]
Saving model.layers.29.linear_attn.conv1d.weight (f32, [8192,1,4]): 42%|████▏ | 232/557 [00:00<00:00, 2958.24it/s]
Saving model.layers.29.linear_attn.conv1d.bias (f32, [8192]): 42%|████▏ | 233/557 [00:00<00:00, 2968.86it/s]
Saving model.layers.29.linear_attn.dt_bias (f32, [32]): 42%|████▏ | 234/557 [00:00<00:00, 2980.25it/s]
Saving model.layers.29.linear_attn.neg_exp_A (f32, [32]): 42%|████▏ | 235/557 [00:00<00:00, 2992.18it/s]
Saving model.layers.29.linear_attn.norm.weight (f32, [128]): 42%|████▏ | 236/557 [00:00<00:00, 3004.03it/s]
Saving model.layers.29.post_attention_layernorm.weight (f32, [2560]): 43%|████▎ | 237/557 [00:00<00:00, 3015.93it/s]
Saving model.layers.30.input_layernorm.weight (f32, [2560]): 43%|████▎ | 238/557 [00:00<00:00, 3027.53it/s]
Saving model.layers.30.linear_attn.b_proj.MatMul.weight (f32, [2560,32]): 43%|████▎ | 239/557 [00:00<00:00, 3039.17it/s]
Saving model.layers.30.linear_attn.a_proj.MatMul.weight (f32, [2560,32]): 43%|████▎ | 240/557 [00:00<00:00, 3048.94it/s]
Saving model.layers.30.linear_attn.conv1d.weight (f32, [8192,1,4]): 43%|████▎ | 241/557 [00:00<00:00, 3058.55it/s]
Saving model.layers.30.linear_attn.conv1d.bias (f32, [8192]): 43%|████▎ | 242/557 [00:00<00:00, 3069.38it/s]
Saving model.layers.30.linear_attn.dt_bias (f32, [32]): 44%|████▎ | 243/557 [00:00<00:00, 3080.65it/s]
Saving model.layers.30.linear_attn.neg_exp_A (f32, [32]): 44%|████▍ | 244/557 [00:00<00:00, 3092.54it/s]
Saving model.layers.30.linear_attn.norm.weight (f32, [128]): 44%|████▍ | 245/557 [00:00<00:00, 3104.31it/s]
Saving model.layers.30.post_attention_layernorm.weight (f32, [2560]): 44%|████▍ | 246/557 [00:00<00:00, 3116.09it/s]
Saving model.layers.31.input_layernorm.weight (f32, [2560]): 44%|████▍ | 247/557 [00:00<00:00, 3127.58it/s]
Saving model.layers.31.attn.q_norm.layernorm.weight (f32, [256]): 45%|████▍ | 248/557 [00:00<00:00, 3139.13it/s]
Saving model.layers.31.attn.k_norm.layernorm.weight (f32, [256]): 45%|████▍ | 249/557 [00:00<00:00, 3150.95it/s]
Saving model.layers.31.post_attention_layernorm.weight (f32, [2560]): 45%|████▍ | 250/557 [00:00<00:00, 3162.60it/s]
Saving model.layers.32.final_norm_layernorm.weight (f32, [2560]): 45%|████▌ | 251/557 [00:00<00:00, 3174.11it/s]
Saving model.layers.0.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 45%|████▌ | 252/557 [00:00<00:00, 3185.61it/s]
Saving model.layers.0.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 45%|████▌ | 253/557 [00:00<00:00, 3142.49it/s]
Saving model.layers.0.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 46%|████▌ | 254/557 [00:00<00:00, 3141.36it/s]
Saving model.layers.0.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 46%|████▌ | 255/557 [00:00<00:00, 3118.58it/s]
Saving model.layers.0.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 46%|████▌ | 256/557 [00:00<00:00, 3110.02it/s]
Saving model.layers.0.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 46%|████▌ | 257/557 [00:00<00:00, 3097.67it/s]
Saving model.layers.0.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 46%|████▋ | 258/557 [00:00<00:00, 3102.85it/s]
Saving model.layers.0.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 46%|████▋ | 259/557 [00:00<00:00, 2989.82it/s]
Saving model.layers.0.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 47%|████▋ | 260/557 [00:00<00:00, 2964.63it/s]
Saving model.layers.0.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 47%|████▋ | 261/557 [00:00<00:00, 2918.34it/s]
Saving model.layers.0.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 47%|████▋ | 262/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 47%|████▋ | 262/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 47%|████▋ | 263/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 47%|████▋ | 264/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 48%|████▊ | 265/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 48%|████▊ | 266/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 48%|████▊ | 267/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 48%|████▊ | 268/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 48%|████▊ | 269/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 48%|████▊ | 270/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.1.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 49%|████▊ | 271/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 49%|████▉ | 272/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 49%|████▉ | 273/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 49%|████▉ | 274/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 49%|████▉ | 275/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 50%|████▉ | 276/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 50%|████▉ | 277/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 50%|████▉ | 278/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 50%|█████ | 279/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 50%|█████ | 280/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.2.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 50%|█████ | 281/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.3.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 51%|█████ | 282/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.3.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 51%|█████ | 283/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.3.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 51%|█████ | 284/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.3.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 51%|█████ | 285/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.3.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 51%|█████▏ | 286/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.3.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 52%|█████▏ | 287/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.3.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 52%|█████▏ | 288/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.3.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 52%|█████▏ | 289/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 52%|█████▏ | 290/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 52%|█████▏ | 291/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 52%|█████▏ | 292/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 53%|█████▎ | 293/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 53%|█████▎ | 294/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 53%|█████▎ | 295/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 53%|█████▎ | 296/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 53%|█████▎ | 297/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 54%|█████▎ | 298/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.4.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 54%|█████▎ | 299/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 54%|█████▍ | 300/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 54%|█████▍ | 301/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 54%|█████▍ | 302/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 54%|█████▍ | 303/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 55%|█████▍ | 304/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 55%|█████▍ | 305/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 55%|█████▍ | 306/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 55%|█████▌ | 307/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 55%|█████▌ | 308/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.5.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 55%|█████▌ | 309/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 56%|█████▌ | 310/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 56%|█████▌ | 311/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 56%|█████▌ | 312/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 56%|█████▌ | 313/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 56%|█████▋ | 314/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 57%|█████▋ | 315/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 57%|█████▋ | 316/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 57%|█████▋ | 317/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 57%|█████▋ | 318/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.6.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 57%|█████▋ | 319/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.7.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 57%|█████▋ | 320/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.7.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 58%|█████▊ | 321/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.7.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 58%|█████▊ | 322/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.7.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 58%|█████▊ | 323/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.7.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 58%|█████▊ | 324/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.7.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 58%|█████▊ | 325/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.7.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 59%|█████▊ | 326/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.7.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 59%|█████▊ | 327/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 59%|█████▉ | 328/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 59%|█████▉ | 329/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 59%|█████▉ | 330/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 59%|█████▉ | 331/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 60%|█████▉ | 332/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 60%|█████▉ | 333/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 60%|█████▉ | 334/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 60%|██████ | 335/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 60%|██████ | 336/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.8.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 61%|██████ | 337/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 61%|██████ | 338/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 61%|██████ | 339/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 61%|██████ | 340/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 61%|██████ | 341/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 61%|██████▏ | 342/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 62%|██████▏ | 343/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 62%|██████▏ | 344/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 62%|██████▏ | 345/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 62%|██████▏ | 346/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.9.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 62%|██████▏ | 347/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 62%|██████▏ | 348/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 63%|██████▎ | 349/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 63%|██████▎ | 350/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 63%|██████▎ | 351/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 63%|██████▎ | 352/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 63%|██████▎ | 353/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 64%|██████▎ | 354/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 64%|██████▎ | 355/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 64%|██████▍ | 356/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.10.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 64%|██████▍ | 357/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.11.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 64%|██████▍ | 358/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.11.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 64%|██████▍ | 359/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.11.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 65%|██████▍ | 360/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.11.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 65%|██████▍ | 361/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.11.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 65%|██████▍ | 362/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.11.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 65%|██████▌ | 363/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.11.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 65%|██████▌ | 364/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.11.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 66%|██████▌ | 365/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 66%|██████▌ | 366/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 66%|██████▌ | 367/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 66%|██████▌ | 368/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 66%|██████▌ | 369/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 66%|██████▋ | 370/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 67%|██████▋ | 371/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 67%|██████▋ | 372/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 67%|██████▋ | 373/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 67%|██████▋ | 374/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.12.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 67%|██████▋ | 375/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 68%|██████▊ | 376/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 68%|██████▊ | 377/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 68%|██████▊ | 378/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 68%|██████▊ | 379/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 68%|██████▊ | 380/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 68%|██████▊ | 381/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 69%|██████▊ | 382/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 69%|██████▉ | 383/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 69%|██████▉ | 384/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.13.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 69%|██████▉ | 385/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 69%|██████▉ | 386/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 69%|██████▉ | 387/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 70%|██████▉ | 388/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 70%|██████▉ | 389/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 70%|███████ | 390/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 70%|███████ | 391/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 70%|███████ | 392/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 71%|███████ | 393/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 71%|███████ | 394/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.14.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 71%|███████ | 395/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.15.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 71%|███████ | 396/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.15.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 71%|███████▏ | 397/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.15.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 71%|███████▏ | 398/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.15.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 72%|███████▏ | 399/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.15.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 72%|███████▏ | 400/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.15.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 72%|███████▏ | 401/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.15.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 72%|███████▏ | 402/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.15.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 72%|███████▏ | 403/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 73%|███████▎ | 404/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 73%|███████▎ | 405/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 73%|███████▎ | 406/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 73%|███████▎ | 407/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 73%|███████▎ | 408/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 73%|███████▎ | 409/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 74%|███████▎ | 410/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 74%|███████▍ | 411/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 74%|███████▍ | 412/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.16.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 74%|███████▍ | 413/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 74%|███████▍ | 414/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 75%|███████▍ | 415/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 75%|███████▍ | 416/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 75%|███████▍ | 417/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 75%|███████▌ | 418/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 75%|███████▌ | 419/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 75%|███████▌ | 420/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 76%|███████▌ | 421/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 76%|███████▌ | 422/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.17.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 76%|███████▌ | 423/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 76%|███████▌ | 424/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 76%|███████▋ | 425/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 76%|███████▋ | 426/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 77%|███████▋ | 427/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 77%|███████▋ | 428/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 77%|███████▋ | 429/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 77%|███████▋ | 430/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 77%|███████▋ | 431/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 78%|███████▊ | 432/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.18.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 78%|███████▊ | 433/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.19.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 78%|███████▊ | 434/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.19.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 78%|███████▊ | 435/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.19.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 78%|███████▊ | 436/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.19.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 78%|███████▊ | 437/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.19.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 79%|███████▊ | 438/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.19.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 79%|███████▉ | 439/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.19.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 79%|███████▉ | 440/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.19.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 79%|███████▉ | 441/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 79%|███████▉ | 442/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 80%|███████▉ | 443/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 80%|███████▉ | 444/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 80%|███████▉ | 445/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 80%|████████ | 446/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 80%|████████ | 447/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 80%|████████ | 448/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 81%|████████ | 449/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 81%|████████ | 450/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.20.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 81%|████████ | 451/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 81%|████████ | 452/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 81%|████████▏ | 453/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 82%|████████▏ | 454/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 82%|████████▏ | 455/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 82%|████████▏ | 456/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 82%|████████▏ | 457/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 82%|████████▏ | 458/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 82%|████████▏ | 459/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 83%|████████▎ | 460/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.21.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 83%|████████▎ | 461/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 83%|████████▎ | 462/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 83%|████████▎ | 463/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 83%|████████▎ | 464/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 83%|████████▎ | 465/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 84%|████████▎ | 466/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 84%|████████▍ | 467/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 84%|████████▍ | 468/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 84%|████████▍ | 469/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 84%|████████▍ | 470/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.22.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 85%|████████▍ | 471/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.23.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 85%|████████▍ | 472/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.23.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 85%|████████▍ | 473/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.23.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 85%|████████▌ | 474/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.23.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 85%|████████▌ | 475/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.23.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 85%|████████▌ | 476/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.23.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 86%|████████▌ | 477/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.23.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 86%|████████▌ | 478/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.23.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 86%|████████▌ | 479/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 86%|████████▌ | 480/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 86%|████████▋ | 481/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 87%|████████▋ | 482/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 87%|████████▋ | 483/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 87%|████████▋ | 484/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 87%|████████▋ | 485/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 87%|████████▋ | 486/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 87%|████████▋ | 487/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 88%|████████▊ | 488/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.24.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 88%|████████▊ | 489/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.25.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 88%|████████▊ | 490/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.25.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 88%|████████▊ | 491/557 [00:00<00:00, 2292.53it/s]
Saving model.layers.25.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 88%|████████▊ | 492/557 [00:00<00:00, 608.59it/s]
Saving model.layers.25.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 88%|████████▊ | 492/557 [00:00<00:00, 608.59it/s]
Saving model.layers.25.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 89%|████████▊ | 493/557 [00:00<00:00, 608.59it/s]
Saving model.layers.25.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 89%|████████▊ | 494/557 [00:00<00:00, 608.59it/s]
Saving model.layers.25.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 89%|████████▉ | 495/557 [00:00<00:00, 608.59it/s]
Saving model.layers.25.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 89%|████████▉ | 496/557 [00:00<00:00, 608.59it/s]
Saving model.layers.25.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 89%|████████▉ | 497/557 [00:00<00:00, 608.59it/s]
Saving model.layers.25.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 89%|████████▉ | 498/557 [00:00<00:00, 608.59it/s]
Saving model.layers.25.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 90%|████████▉ | 499/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 90%|████████▉ | 500/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 90%|████████▉ | 501/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 90%|█████████ | 502/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 90%|█████████ | 503/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 90%|█████████ | 504/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 91%|█████████ | 505/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 91%|█████████ | 506/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 91%|█████████ | 507/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 91%|█████████ | 508/557 [00:00<00:00, 608.59it/s]
Saving model.layers.26.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 91%|█████████▏| 509/557 [00:00<00:00, 608.59it/s]
Saving model.layers.27.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 92%|█████████▏| 510/557 [00:00<00:00, 608.59it/s]
Saving model.layers.27.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 92%|█████████▏| 511/557 [00:00<00:00, 608.59it/s]
Saving model.layers.27.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 92%|█████████▏| 512/557 [00:00<00:00, 608.59it/s]
Saving model.layers.27.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 92%|█████████▏| 513/557 [00:00<00:00, 608.59it/s]
Saving model.layers.27.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 92%|█████████▏| 514/557 [00:00<00:00, 608.59it/s]
Saving model.layers.27.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 92%|█████████▏| 515/557 [00:00<00:00, 608.59it/s]
Saving model.layers.27.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 93%|█████████▎| 516/557 [00:00<00:00, 608.59it/s]
Saving model.layers.27.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 93%|█████████▎| 517/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 93%|█████████▎| 518/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 93%|█████████▎| 519/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 93%|█████████▎| 520/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 94%|█████████▎| 521/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 94%|█████████▎| 522/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 94%|█████████▍| 523/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 94%|█████████▍| 524/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 94%|█████████▍| 525/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 94%|█████████▍| 526/557 [00:00<00:00, 608.59it/s]
Saving model.layers.28.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 95%|█████████▍| 527/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 95%|█████████▍| 528/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 95%|█████████▍| 529/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 95%|█████████▌| 530/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 95%|█████████▌| 531/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 96%|█████████▌| 532/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 96%|█████████▌| 533/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 96%|█████████▌| 534/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 96%|█████████▌| 535/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 96%|█████████▌| 536/557 [00:00<00:00, 608.59it/s]
Saving model.layers.29.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 96%|█████████▋| 537/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.linear_attn.qkv_proj.MatMul.weight_Q4 (u8, [8192,80,16]): 97%|█████████▋| 538/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.linear_attn.qkv_proj.MatMul.weight_scales (f32, [8192,80]): 97%|█████████▋| 539/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.linear_attn.z_proj.MatMul.weight_Q4 (u8, [4096,80,16]): 97%|█████████▋| 540/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.linear_attn.z_proj.MatMul.weight_scales (f32, [4096,80]): 97%|█████████▋| 541/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.linear_attn.out_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 97%|█████████▋| 542/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.linear_attn.out_proj.MatMul.weight_scales (f32, [2560,128]): 97%|█████████▋| 543/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 98%|█████████▊| 544/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 98%|█████████▊| 545/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 98%|█████████▊| 546/557 [00:00<00:00, 608.59it/s]
Saving model.layers.30.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 98%|█████████▊| 547/557 [00:00<00:00, 608.59it/s]
Saving model.layers.31.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 98%|█████████▊| 548/557 [00:00<00:00, 608.59it/s]
Saving model.layers.31.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 99%|█████████▊| 549/557 [00:00<00:00, 608.59it/s]
Saving model.layers.31.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 99%|█████████▊| 550/557 [00:00<00:00, 608.59it/s]
Saving model.layers.31.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 99%|█████████▉| 551/557 [00:00<00:00, 608.59it/s]
Saving model.layers.31.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 99%|█████████▉| 552/557 [00:00<00:00, 608.59it/s]
Saving model.layers.31.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 99%|█████████▉| 553/557 [00:00<00:00, 608.59it/s]
Saving model.layers.31.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 99%|█████████▉| 554/557 [00:00<00:00, 608.59it/s]
Saving model.layers.31.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 100%|█████████▉| 555/557 [00:00<00:00, 608.59it/s]
Saving lm_head.MatMul.weight_Q4 (u8, [248320,80,16]): 100%|█████████▉| 556/557 [00:00<00:00, 608.59it/s]
Saving lm_head.MatMul.weight_scales (f32, [248320,80]): 100%|██████████| 557/557 [00:01<00:00, 608.59it/s]
Saving lm_head.MatMul.weight_scales (f32, [248320,80]): 100%|██████████| 557/557 [00:01<00:00, 503.27it/s] |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[1] ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/ReduceSum ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/Sub ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/Sub/Cast ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/Shape ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/1 ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/attn_mask_reformat/attn_mask_subgraph/Gather ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - Gather doesn't have const weight. Skip quantization. |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/attn_mask_reformat/attn_mask_subgraph/Gather with 4 bits ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/attn_mask_reformat/attn_mask_subgraph/Gather/Cast ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[248320, 1280] ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/mtp/embed_tokens/Reshape ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/mtp/embed_tokens/GatherBlockQuantized ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/mtp/pre_fc_norm_embedding ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/mtp/pre_fc_norm_hidden ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/mtp/fc/Concat ... |
| 2026-11-05 03:43:03,790 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/mtp/fc/MatMul ... |
| 2026-11-05 03:43:03,823 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/mtp/fc/MatMul with 4 bits ... |
| 2026-11-05 03:43:03,823 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/input_layernorm/LayerNorm ... |
| 2026-11-05 03:43:03,823 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/attn/qkv_proj/MatMul ... |
| 2026-11-05 03:43:03,870 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/attn/qkv_proj/MatMul with 4 bits ... |
| 2026-11-05 03:43:03,870 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[8192, 1024, 1024] ... |
| 2026-11-05 03:43:03,870 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/qkv_proj/Split ... |
| 2026-11-05 03:43:03,870 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, 0, 16, 512] ... |
| 2026-11-05 03:43:03,870 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/q_gate/Reshape ... |
| 2026-11-05 03:43:03,870 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[256, 256] ... |
| 2026-11-05 03:43:03,870 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/q_gate/Split ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, 0, 4096] ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/q_proj/Reshape ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/gate/Reshape ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, -1, 256] ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/q_norm/Reshape_1 ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/q_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, -1, 4096] ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/q_norm/Reshape_2 ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/k_norm/Reshape_1 ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/k_norm/SimplifiedLayerNormalization ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/INT64/[0, -1, 1024] ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/k_norm/Reshape_2 ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/q_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/k_rotary/MRotaryEmbedding ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/GroupQueryAttention ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/gate/Sigmoid ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/attn/gate/Mul ... |
| 2026-11-05 03:43:03,871 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/attn/o_proj/MatMul ... |
| 2026-11-05 03:43:03,896 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/attn/o_proj/MatMul with 4 bits ... |
| 2026-11-05 03:43:03,896 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/post_attention_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:43:03,896 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/mlp/gate_up_proj/MatMul ... |
| 2026-11-05 03:43:04,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/mlp/gate_up_proj/MatMul with 4 bits ... |
| 2026-11-05 03:43:04,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/mlp/gate_up_proj/Split ... |
| 2026-11-05 03:43:04,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/mlp/act_fn/Sigmoid ... |
| 2026-11-05 03:43:04,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/mlp/act_fn/Mul ... |
| 2026-11-05 03:43:04,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.0/mlp/Mul ... |
| 2026-11-05 03:43:04,022 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /model/layers.0/mlp/down_proj/MatMul ... |
| 2026-11-05 03:43:04,070 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /model/layers.0/mlp/down_proj/MatMul with 4 bits ... |
| 2026-11-05 03:43:04,070 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/layers.1/final_norm_layernorm/SkipLayerNorm ... |
| 2026-11-05 03:43:04,070 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - start to quantize /lm_head/MatMul ... |
| 2026-11-05 03:43:06,450 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - complete quantization of /lm_head/MatMul with 4 bits ... |
| Saving ONNX model in /home/anon/knowline/onnx/model-cpu |
|
0it [00:00, ?it/s]
Saving model.mtp.pre_fc_norm_embedding.weight (f32, [2560]): 5%|▍ | 1/21 [00:00<00:00, 735.46it/s]
Saving model.mtp.pre_fc_norm_hidden.weight (f32, [2560]): 10%|▉ | 2/21 [00:00<00:00, 1372.71it/s]
Saving model.layers.0.input_layernorm.weight (f32, [2560]): 14%|█▍ | 3/21 [00:00<00:00, 1997.92it/s]
Saving model.layers.0.attn.q_norm.layernorm.weight (f32, [256]): 19%|█▉ | 4/21 [00:00<00:00, 2603.94it/s]
Saving model.layers.0.attn.k_norm.layernorm.weight (f32, [256]): 24%|██▍ | 5/21 [00:00<00:00, 3199.80it/s]
Saving cos_cache (f32, [262144,32]): 29%|██▊ | 6/21 [00:00<00:00, 3772.99it/s]
Saving sin_cache (f32, [262144,32]): 33%|███▎ | 7/21 [00:00<00:00, 1170.66it/s]
Saving model.layers.0.post_attention_layernorm.weight (f32, [2560]): 38%|███▊ | 8/21 [00:00<00:00, 758.81it/s]
Saving model.layers.1.final_norm_layernorm.weight (f32, [2560]): 43%|████▎ | 9/21 [00:00<00:00, 845.23it/s]
Saving model.mtp.fc.MatMul.weight_Q4 (u8, [2560,160,16]): 48%|████▊ | 10/21 [00:00<00:00, 935.04it/s]
Saving model.mtp.fc.MatMul.weight_scales (f32, [2560,160]): 52%|█████▏ | 11/21 [00:00<00:00, 953.27it/s]
Saving model.layers.0.attn.qkv_proj.MatMul.weight_Q4 (u8, [10240,80,16]): 57%|█████▋ | 12/21 [00:00<00:00, 1019.48it/s]
Saving model.layers.0.attn.qkv_proj.MatMul.weight_scales (f32, [10240,80]): 62%|██████▏ | 13/21 [00:00<00:00, 961.08it/s]
Saving model.layers.0.attn.o_proj.MatMul.weight_Q4 (u8, [2560,128,16]): 67%|██████▋ | 14/21 [00:00<00:00, 986.23it/s]
Saving model.layers.0.attn.o_proj.MatMul.weight_scales (f32, [2560,128]): 71%|███████▏ | 15/21 [00:00<00:00, 1003.15it/s]
Saving model.layers.0.mlp.gate_up_proj.MatMul.input_1_Q4 (u8, [18432,80,16]): 76%|███████▌ | 16/21 [00:00<00:00, 1055.02it/s]
Saving model.layers.0.mlp.gate_up_proj.MatMul.input_1_scales (f32, [18432,80]): 81%|████████ | 17/21 [00:00<00:00, 932.48it/s]
Saving model.layers.0.mlp.down_proj.MatMul.weight_Q4 (u8, [2560,288,16]): 86%|████████▌ | 18/21 [00:00<00:00, 946.45it/s]
Saving model.layers.0.mlp.down_proj.MatMul.weight_scales (f32, [2560,288]): 90%|█████████ | 19/21 [00:00<00:00, 915.77it/s]
Saving lm_head.MatMul.weight_Q4 (u8, [248320,80,16]): 95%|█████████▌| 20/21 [00:00<00:00, 945.72it/s]
Saving lm_head.MatMul.weight_Q4 (u8, [248320,80,16]): 100%|██████████| 21/21 [00:00<00:00, 131.28it/s]
Saving lm_head.MatMul.weight_scales (f32, [248320,80]): 100%|██████████| 21/21 [00:00<00:00, 131.28it/s]
Saving lm_head.MatMul.weight_scales (f32, [248320,80]): 100%|██████████| 21/21 [00:00<00:00, 114.16it/s] |
| Configured 397 MB of runtime initializer sharing; /home/anon/knowline/onnx/model-cpu/mtp.onnx.data remains self-contained. |
| Warning: could not read generation_config.json (/home/anon/LLM/KnowLine-4B-Gen5.5 does not appear to have a file named generation_config.json. Checkout 'https://huggingface.co//home/anon/LLM/KnowLine-4B-Gen5.5/tree/main' for available files.). Falling back to config.json. |
| Adding the tokenizer's end-of-turn token <|im_end|> (id 248046) to eos_token_id from the model configuration: 248044. |
| Saving GenAI config in /home/anon/knowline/onnx/model-cpu |
| Added 'mtp' section to genai_config.json |
| Saving processing files in /home/anon/knowline/onnx/model-cpu for GenAI |
| EXIT:0 |
| |
