Image-Text-to-Text
Safetensors
English
Thai
Chinese
qwen3_5
qwen3.8
qwen3.8-27b
daisyrose
gptq
int4
4-bit precision
agents
mtp
speculative-decoding
conversational
vision
4-bit precision
Instructions to use palmfuture/DaisyRose-27B-GPTQ-Int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Local Apps Settings
- vLLM
How to use palmfuture/DaisyRose-27B-GPTQ-Int4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "palmfuture/DaisyRose-27B-GPTQ-Int4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "palmfuture/DaisyRose-27B-GPTQ-Int4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/palmfuture/DaisyRose-27B-GPTQ-Int4
- SGLang
How to use palmfuture/DaisyRose-27B-GPTQ-Int4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "palmfuture/DaisyRose-27B-GPTQ-Int4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "palmfuture/DaisyRose-27B-GPTQ-Int4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "palmfuture/DaisyRose-27B-GPTQ-Int4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "palmfuture/DaisyRose-27B-GPTQ-Int4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use palmfuture/DaisyRose-27B-GPTQ-Int4 with Docker Model Runner:
docker model run hf.co/palmfuture/DaisyRose-27B-GPTQ-Int4
Download quant_log.csv from palmfuture/DaisyRose-27B-GPTQ-Int4: direct link, hf CLI and curl.
- Browser
- Download file 19.2 kB
-
https://huggingface.co/palmfuture/DaisyRose-27B-GPTQ-Int4/resolve/main/quant_log.csv
- Command line
-
hf download hf://palmfuture/DaisyRose-27B-GPTQ-Int4/quant_log.csv
-
curl -L -o quant_log.csv https://huggingface.co/palmfuture/DaisyRose-27B-GPTQ-Int4/resolve/main/quant_log.csv
19.2 kB
| layer,module,loss,samples,damp,time | |
| 0,linear_attn.in_proj_qkv,0.0001415895,0.05000,2.373 | |
| 0,linear_attn.in_proj_z,0.0000908838,0.05000,1.890 | |
| 0,linear_attn.out_proj,0.0000000979,0.05000,2.135 | |
| 0,mlp.up_proj,0.0000008764,0.05000,2.505 | |
| 0,mlp.gate_proj,0.0000009462,0.05000,2.559 | |
| 0,mlp.down_proj,0.0000000099,0.05000,8.625 | |
| 1,linear_attn.in_proj_qkv,0.0000065204,0.05000,1.744 | |
| 1,linear_attn.in_proj_z,0.0000039944,0.05000,1.643 | |
| 1,linear_attn.out_proj,0.0000000254,0.05000,2.059 | |
| 1,mlp.gate_proj,0.0000019041,0.05000,2.495 | |
| 1,mlp.up_proj,0.0000018170,0.05000,2.530 | |
| 1,mlp.down_proj,0.0000000168,0.05000,8.464 | |
| 2,linear_attn.in_proj_qkv,0.0000094563,0.05000,1.897 | |
| 2,linear_attn.in_proj_z,0.0000057184,0.05000,1.722 | |
| 2,linear_attn.out_proj,0.0000000409,0.05000,2.125 | |
| 2,mlp.up_proj,0.0000032307,0.05000,2.596 | |
| 2,mlp.gate_proj,0.0000034303,0.05000,2.619 | |
| 2,mlp.down_proj,0.0000000476,0.05000,8.424 | |
| 3,self_attn.k_proj,0.0000067441,0.05000,3.521 | |
| 3,self_attn.v_proj,0.0000066433,0.05000,3.525 | |
| 3,self_attn.q_proj,0.0000960108,0.05000,3.663 | |
| 3,self_attn.o_proj,0.0000000326,0.05000,2.069 | |
| 3,mlp.gate_proj,0.0000040240,0.05000,2.466 | |
| 3,mlp.up_proj,0.0000038741,0.05000,2.553 | |
| 3,mlp.down_proj,0.0000000401,0.05000,8.385 | |
| 4,linear_attn.in_proj_qkv,0.0000125690,0.05000,1.854 | |
| 4,linear_attn.in_proj_z,0.0000080093,0.05000,1.733 | |
| 4,linear_attn.out_proj,0.0000000481,0.05000,1.974 | |
| 4,mlp.up_proj,0.0000049819,0.05000,2.519 | |
| 4,mlp.gate_proj,0.0000051489,0.05000,2.533 | |
| 4,mlp.down_proj,0.0000000510,0.05000,8.373 | |
| 5,linear_attn.in_proj_qkv,0.0000158064,0.05000,1.732 | |
| 5,linear_attn.in_proj_z,0.0000102854,0.05000,1.969 | |
| 5,linear_attn.out_proj,0.0000000680,0.05000,2.024 | |
| 5,mlp.up_proj,0.0000063387,0.05000,2.477 | |
| 5,mlp.gate_proj,0.0000067229,0.05000,2.522 | |
| 5,mlp.down_proj,0.0000000926,0.05000,8.138 | |
| 6,linear_attn.in_proj_qkv,0.0000240834,0.05000,1.842 | |
| 6,linear_attn.in_proj_z,0.0000144446,0.05000,1.765 | |
| 6,linear_attn.out_proj,0.0000001000,0.05000,2.064 | |
| 6,mlp.up_proj,0.0000082548,0.05000,2.485 | |
| 6,mlp.gate_proj,0.0000088471,0.05000,2.578 | |
| 6,mlp.down_proj,0.0000001448,0.05000,8.556 | |
| 7,self_attn.v_proj,0.0000057773,0.05000,3.445 | |
| 7,self_attn.k_proj,0.0000062458,0.05000,3.498 | |
| 7,self_attn.q_proj,0.0000801214,0.05000,3.577 | |
| 7,self_attn.o_proj,0.0000000641,0.05000,2.066 | |
| 7,mlp.gate_proj,0.0000096985,0.05000,2.515 | |
| 7,mlp.up_proj,0.0000090513,0.05000,2.570 | |
| 7,mlp.down_proj,0.0000001563,0.05000,8.473 | |
| 8,linear_attn.in_proj_qkv,0.0000238553,0.05000,1.791 | |
| 8,linear_attn.in_proj_z,0.0000150331,0.05000,1.707 | |
| 8,linear_attn.out_proj,0.0000001302,0.05000,2.002 | |
| 8,mlp.gate_proj,0.0000105906,0.05000,2.485 | |
| 8,mlp.up_proj,0.0000099658,0.05000,2.502 | |
| 8,mlp.down_proj,0.0000001758,0.05000,8.618 | |
| 9,linear_attn.in_proj_qkv,0.0000243231,0.05000,1.764 | |
| 9,linear_attn.in_proj_z,0.0000148245,0.05000,1.703 | |
| 9,linear_attn.out_proj,0.0000001364,0.05000,2.091 | |
| 9,mlp.up_proj,0.0000106785,0.05000,2.458 | |
| 9,mlp.gate_proj,0.0000112768,0.05000,2.529 | |
| 9,mlp.down_proj,0.0000001879,0.05000,8.526 | |
| 10,linear_attn.in_proj_qkv,0.0000249240,0.05000,1.733 | |
| 10,linear_attn.in_proj_z,0.0000147703,0.05000,1.670 | |
| 10,linear_attn.out_proj,0.0000001592,0.05000,2.136 | |
| 10,mlp.gate_proj,0.0000112795,0.05000,2.490 | |
| 10,mlp.up_proj,0.0000107451,0.05000,2.510 | |
| 10,mlp.down_proj,0.0000001973,0.05000,8.688 | |
| 11,self_attn.k_proj,0.0000056570,0.05000,3.504 | |
| 11,self_attn.v_proj,0.0000051878,0.05000,3.524 | |
| 11,self_attn.q_proj,0.0000679004,0.05000,3.598 | |
| 11,self_attn.o_proj,0.0000001081,0.05000,2.025 | |
| 11,mlp.up_proj,0.0000112788,0.05000,2.495 | |
| 11,mlp.gate_proj,0.0000117931,0.05000,2.515 | |
| 11,mlp.down_proj,0.0000002219,0.05000,8.474 | |
| 12,linear_attn.in_proj_qkv,0.0000276019,0.05000,1.748 | |
| 12,linear_attn.in_proj_z,0.0000163925,0.05000,1.653 | |
| 12,linear_attn.out_proj,0.0000002310,0.05000,2.068 | |
| 12,mlp.up_proj,0.0000115713,0.05000,2.483 | |
| 12,mlp.gate_proj,0.0000121107,0.05000,2.525 | |
| 12,mlp.down_proj,0.0000002493,0.05000,8.336 | |
| 13,linear_attn.in_proj_qkv,0.0000289128,0.05000,1.751 | |
| 13,linear_attn.in_proj_z,0.0000163752,0.05000,1.683 | |
| 13,linear_attn.out_proj,0.0000002423,0.05000,2.070 | |
| 13,mlp.up_proj,0.0000122729,0.05000,2.497 | |
| 13,mlp.gate_proj,0.0000128371,0.05000,2.499 | |
| 13,mlp.down_proj,0.0000002771,0.05000,8.415 | |
| 14,linear_attn.in_proj_qkv,0.0000314222,0.05000,1.746 | |
| 14,linear_attn.in_proj_z,0.0000183053,0.05000,1.702 | |
| 14,linear_attn.out_proj,0.0000002906,0.05000,2.017 | |
| 14,mlp.up_proj,0.0000128540,0.05000,2.474 | |
| 14,mlp.gate_proj,0.0000134083,0.05000,2.520 | |
| 14,mlp.down_proj,0.0000003008,0.05000,8.424 | |
| 15,self_attn.k_proj,0.0000048910,0.05000,3.519 | |
| 15,self_attn.v_proj,0.0000047812,0.05000,3.572 | |
| 15,self_attn.q_proj,0.0000562981,0.05000,3.616 | |
| 15,self_attn.o_proj,0.0000001822,0.05000,1.999 | |
| 15,mlp.gate_proj,0.0000151246,0.05000,2.521 | |
| 15,mlp.up_proj,0.0000146288,0.05000,2.595 | |
| 15,mlp.down_proj,0.0000003547,0.05000,8.768 | |
| 16,linear_attn.in_proj_qkv,0.0000328558,0.05000,1.724 | |
| 16,linear_attn.in_proj_z,0.0000187999,0.05000,1.668 | |
| 16,linear_attn.out_proj,0.0000003097,0.05000,2.059 | |
| 16,mlp.up_proj,0.0000160764,0.05000,2.483 | |
| 16,mlp.gate_proj,0.0000167138,0.05000,2.551 | |
| 16,mlp.down_proj,0.0000004356,0.05000,8.380 | |
| 17,linear_attn.in_proj_qkv,0.0000374627,0.05000,1.737 | |
| 17,linear_attn.in_proj_z,0.0000193338,0.05000,1.776 | |
| 17,linear_attn.out_proj,0.0000004120,0.05000,1.981 | |
| 17,mlp.gate_proj,0.0000191973,0.05000,2.463 | |
| 17,mlp.up_proj,0.0000184522,0.05000,2.536 | |
| 17,mlp.down_proj,0.0000005747,0.05000,8.469 | |
| 18,linear_attn.in_proj_qkv,0.0000405164,0.05000,1.776 | |
| 18,linear_attn.in_proj_z,0.0000225120,0.05000,1.637 | |
| 18,linear_attn.out_proj,0.0000005003,0.05000,2.082 | |
| 18,mlp.up_proj,0.0000221460,0.05000,2.367 | |
| 18,mlp.gate_proj,0.0000235847,0.05000,2.693 | |
| 18,mlp.down_proj,0.0000009146,0.05000,8.442 | |
| 19,self_attn.v_proj,0.0000068581,0.05000,3.590 | |
| 19,self_attn.k_proj,0.0000060692,0.05000,3.596 | |
| 19,self_attn.q_proj,0.0000679467,0.05000,3.650 | |
| 19,self_attn.o_proj,0.0000007338,0.05000,2.057 | |
| 19,mlp.gate_proj,0.0000300192,0.05000,2.510 | |
| 19,mlp.up_proj,0.0000284208,0.05000,2.546 | |
| 19,mlp.down_proj,0.0000012716,0.05000,8.322 | |
| 20,linear_attn.in_proj_qkv,0.0000730709,0.05000,1.718 | |
| 20,linear_attn.in_proj_z,0.0000412970,0.05000,1.774 | |
| 20,linear_attn.out_proj,0.0000007682,0.05000,2.127 | |
| 20,mlp.up_proj,0.0000343496,0.05000,2.447 | |
| 20,mlp.gate_proj,0.0000369017,0.05000,2.506 | |
| 20,mlp.down_proj,0.0000014915,0.05000,8.484 | |
| 21,linear_attn.in_proj_qkv,0.0000898600,0.05000,1.752 | |
| 21,linear_attn.in_proj_z,0.0000575687,0.05000,1.667 | |
| 21,linear_attn.out_proj,0.0000010031,0.05000,2.067 | |
| 21,mlp.up_proj,0.0000379527,0.05000,2.448 | |
| 21,mlp.gate_proj,0.0000405016,0.05000,2.467 | |
| 21,mlp.down_proj,0.0000016183,0.05000,8.441 | |
| 22,linear_attn.in_proj_qkv,0.0000789031,0.05000,1.831 | |
| 22,linear_attn.in_proj_z,0.0000483614,0.05000,1.691 | |
| 22,linear_attn.out_proj,0.0000009653,0.05000,2.024 | |
| 22,mlp.up_proj,0.0000426656,0.05000,2.470 | |
| 22,mlp.gate_proj,0.0000479491,0.05000,2.530 | |
| 22,mlp.down_proj,0.0000018790,0.05000,8.521 | |
| 23,self_attn.v_proj,0.0000095328,0.05000,3.541 | |
| 23,self_attn.k_proj,0.0000088634,0.05000,3.557 | |
| 23,self_attn.q_proj,0.0000905101,0.05000,3.620 | |
| 23,self_attn.o_proj,0.0000009522,0.05000,2.050 | |
| 23,mlp.up_proj,0.0000465102,0.05000,2.478 | |
| 23,mlp.gate_proj,0.0000500930,0.05000,2.543 | |
| 23,mlp.down_proj,0.0000020293,0.05000,8.477 | |
| 24,linear_attn.in_proj_qkv,0.0000787676,0.05000,1.791 | |
| 24,linear_attn.in_proj_z,0.0000446243,0.05000,1.697 | |
| 24,linear_attn.out_proj,0.0000012415,0.05000,1.995 | |
| 24,mlp.gate_proj,0.0000551030,0.05000,2.453 | |
| 24,mlp.up_proj,0.0000510910,0.05000,2.491 | |
| 24,mlp.down_proj,0.0000023995,0.05000,8.639 | |
| 25,linear_attn.in_proj_qkv,0.0000811426,0.05000,1.743 | |
| 25,linear_attn.in_proj_z,0.0000429533,0.05000,1.633 | |
| 25,linear_attn.out_proj,0.0000013401,0.05000,2.056 | |
| 25,mlp.gate_proj,0.0000536530,0.05000,2.553 | |
| 25,mlp.up_proj,0.0000522607,0.05000,2.607 | |
| 25,mlp.down_proj,0.0000024329,0.05000,8.373 | |
| 26,linear_attn.in_proj_qkv,0.0000737605,0.05000,1.793 | |
| 26,linear_attn.in_proj_z,0.0000375685,0.05000,1.644 | |
| 26,linear_attn.out_proj,0.0000015288,0.05000,1.998 | |
| 26,mlp.gate_proj,0.0000505006,0.05000,2.555 | |
| 26,mlp.up_proj,0.0000516807,0.05000,2.568 | |
| 26,mlp.down_proj,0.0000026131,0.05000,8.312 | |
| 27,self_attn.v_proj,0.0000079873,0.05000,3.466 | |
| 27,self_attn.k_proj,0.0000084741,0.05000,3.583 | |
| 27,self_attn.q_proj,0.0000849405,0.05000,3.671 | |
| 27,self_attn.o_proj,0.0000021771,0.05000,2.044 | |
| 27,mlp.gate_proj,0.0000530773,0.05000,2.520 | |
| 27,mlp.up_proj,0.0000561239,0.05000,2.531 | |
| 27,mlp.down_proj,0.0000028333,0.05000,8.554 | |
| 28,linear_attn.in_proj_qkv,0.0000734093,0.05000,1.753 | |
| 28,linear_attn.in_proj_z,0.0000349343,0.05000,2.133 | |
| 28,linear_attn.out_proj,0.0000019981,0.05000,2.154 | |
| 28,mlp.gate_proj,0.0000513373,0.05000,2.522 | |
| 28,mlp.up_proj,0.0000552970,0.05000,2.719 | |
| 28,mlp.down_proj,0.0000028993,0.05000,8.419 | |
| 29,linear_attn.in_proj_qkv,0.0000800738,0.05000,1.788 | |
| 29,linear_attn.in_proj_z,0.0000370300,0.05000,1.652 | |
| 29,linear_attn.out_proj,0.0000018425,0.05000,2.097 | |
| 29,mlp.up_proj,0.0000560899,0.05000,2.414 | |
| 29,mlp.gate_proj,0.0000517800,0.05000,2.453 | |
| 29,mlp.down_proj,0.0000029493,0.05000,8.329 | |
| 30,linear_attn.in_proj_qkv,0.0000801585,0.05000,1.839 | |
| 30,linear_attn.in_proj_z,0.0000379628,0.05000,1.663 | |
| 30,linear_attn.out_proj,0.0000023410,0.05000,2.043 | |
| 30,mlp.gate_proj,0.0000507298,0.05000,2.516 | |
| 30,mlp.up_proj,0.0000563596,0.05000,2.559 | |
| 30,mlp.down_proj,0.0000030715,0.05000,8.345 | |
| 31,self_attn.k_proj,0.0000086730,0.05000,3.548 | |
| 31,self_attn.v_proj,0.0000102041,0.05000,3.582 | |
| 31,self_attn.q_proj,0.0000832653,0.05000,3.632 | |
| 31,self_attn.o_proj,0.0000031134,0.05000,2.007 | |
| 31,mlp.gate_proj,0.0000545463,0.05000,2.457 | |
| 31,mlp.up_proj,0.0000621353,0.05000,2.470 | |
| 31,mlp.down_proj,0.0000034004,0.05000,8.512 | |
| 32,linear_attn.in_proj_qkv,0.0000857552,0.05000,1.836 | |
| 32,linear_attn.in_proj_z,0.0000379068,0.05000,1.661 | |
| 32,linear_attn.out_proj,0.0000024312,0.05000,2.140 | |
| 32,mlp.up_proj,0.0000667709,0.05000,2.489 | |
| 32,mlp.gate_proj,0.0000581455,0.05000,2.503 | |
| 32,mlp.down_proj,0.0000036503,0.05000,8.387 | |
| 33,linear_attn.in_proj_qkv,0.0001025803,0.05000,1.797 | |
| 33,linear_attn.in_proj_z,0.0000425814,0.05000,1.699 | |
| 33,linear_attn.out_proj,0.0000027966,0.05000,2.039 | |
| 33,mlp.up_proj,0.0000704642,0.05000,2.507 | |
| 33,mlp.gate_proj,0.0000607022,0.05000,2.553 | |
| 33,mlp.down_proj,0.0000037762,0.05000,8.808 | |
| 34,linear_attn.in_proj_qkv,0.0001007167,0.05000,1.840 | |
| 34,linear_attn.in_proj_z,0.0000457376,0.05000,1.860 | |
| 34,linear_attn.out_proj,0.0000028584,0.05000,2.087 | |
| 34,mlp.up_proj,0.0000760321,0.05000,2.506 | |
| 34,mlp.gate_proj,0.0000660409,0.05000,2.546 | |
| 34,mlp.down_proj,0.0000049126,0.05000,8.430 | |
| 35,self_attn.v_proj,0.0000160017,0.05000,3.583 | |
| 35,self_attn.k_proj,0.0000111762,0.05000,3.583 | |
| 35,self_attn.q_proj,0.0001067292,0.05000,3.633 | |
| 35,self_attn.o_proj,0.0000051454,0.05000,2.030 | |
| 35,mlp.gate_proj,0.0000708926,0.05000,2.505 | |
| 35,mlp.up_proj,0.0000800901,0.05000,2.557 | |
| 35,mlp.down_proj,0.0000053974,0.05000,8.634 | |
| 36,linear_attn.in_proj_qkv,0.0001435454,0.05000,1.839 | |
| 36,linear_attn.in_proj_z,0.0000721331,0.05000,1.629 | |
| 36,linear_attn.out_proj,0.0000023714,0.05000,2.155 | |
| 36,mlp.up_proj,0.0000791930,0.05000,2.509 | |
| 36,mlp.gate_proj,0.0000790468,0.05000,2.521 | |
| 36,mlp.down_proj,0.0000047655,0.05000,8.548 | |
| 37,linear_attn.in_proj_qkv,0.0001563138,0.05000,1.743 | |
| 37,linear_attn.in_proj_z,0.0000893192,0.05000,1.629 | |
| 37,linear_attn.out_proj,0.0000025616,0.05000,2.022 | |
| 37,mlp.up_proj,0.0000791695,0.05000,2.466 | |
| 37,mlp.gate_proj,0.0000808207,0.05000,2.481 | |
| 37,mlp.down_proj,0.0000045584,0.05000,8.290 | |
| 38,linear_attn.in_proj_qkv,0.0001404201,0.05000,1.788 | |
| 38,linear_attn.in_proj_z,0.0000820007,0.05000,1.677 | |
| 38,linear_attn.out_proj,0.0000020989,0.05000,2.104 | |
| 38,mlp.up_proj,0.0000897098,0.05000,2.401 | |
| 38,mlp.gate_proj,0.0001057571,0.05000,2.714 | |
| 38,mlp.down_proj,0.0000047199,0.05000,8.452 | |
| 39,self_attn.v_proj,0.0000166644,0.05000,3.468 | |
| 39,self_attn.k_proj,0.0000131877,0.05000,3.561 | |
| 39,self_attn.q_proj,0.0001256809,0.05000,3.577 | |
| 39,self_attn.o_proj,0.0000032104,0.05000,2.039 | |
| 39,mlp.gate_proj,0.0000971474,0.05000,2.510 | |
| 39,mlp.up_proj,0.0000878257,0.05000,2.576 | |
| 39,mlp.down_proj,0.0000045279,0.05000,8.691 | |
| 40,linear_attn.in_proj_qkv,0.0001358290,0.05000,1.753 | |
| 40,linear_attn.in_proj_z,0.0000753472,0.05000,1.670 | |
| 40,linear_attn.out_proj,0.0000024175,0.05000,2.072 | |
| 40,mlp.gate_proj,0.0001051827,0.05000,2.432 | |
| 40,mlp.up_proj,0.0000933012,0.05000,2.445 | |
| 40,mlp.down_proj,0.0000047329,0.05000,8.807 | |
| 41,linear_attn.in_proj_qkv,0.0001247878,0.05000,1.703 | |
| 41,linear_attn.in_proj_z,0.0000682961,0.05000,1.643 | |
| 41,linear_attn.out_proj,0.0000021222,0.05000,2.154 | |
| 41,mlp.gate_proj,0.0000912524,0.05000,2.449 | |
| 41,mlp.up_proj,0.0000899018,0.05000,2.553 | |
| 41,mlp.down_proj,0.0000046516,0.05000,8.431 | |
| 42,linear_attn.in_proj_qkv,0.0001165548,0.05000,1.724 | |
| 42,linear_attn.in_proj_z,0.0000616308,0.05000,1.646 | |
| 42,linear_attn.out_proj,0.0000026365,0.05000,2.032 | |
| 42,mlp.gate_proj,0.0000835011,0.05000,2.437 | |
| 42,mlp.up_proj,0.0000880308,0.05000,2.484 | |
| 42,mlp.down_proj,0.0000051727,0.05000,8.227 | |
| 43,self_attn.k_proj,0.0000132240,0.05000,3.486 | |
| 43,self_attn.v_proj,0.0000168025,0.05000,3.515 | |
| 43,self_attn.q_proj,0.0001228940,0.05000,3.613 | |
| 43,self_attn.o_proj,0.0000047560,0.05000,1.998 | |
| 43,mlp.up_proj,0.0000909114,0.05000,2.439 | |
| 43,mlp.gate_proj,0.0000826320,0.05000,2.472 | |
| 43,mlp.down_proj,0.0000053908,0.05000,8.443 | |
| 44,linear_attn.in_proj_qkv,0.0001129618,0.05000,1.768 | |
| 44,linear_attn.in_proj_z,0.0000551041,0.05000,1.665 | |
| 44,linear_attn.out_proj,0.0000034856,0.05000,2.072 | |
| 44,mlp.up_proj,0.0000923018,0.05000,2.421 | |
| 44,mlp.gate_proj,0.0000830166,0.05000,2.474 | |
| 44,mlp.down_proj,0.0000059648,0.05000,8.337 | |
| 45,linear_attn.in_proj_qkv,0.0001158424,0.05000,1.714 | |
| 45,linear_attn.in_proj_z,0.0000564856,0.05000,1.665 | |
| 45,linear_attn.out_proj,0.0000029727,0.05000,2.095 | |
| 45,mlp.up_proj,0.0000934292,0.05000,2.452 | |
| 45,mlp.gate_proj,0.0000835527,0.05000,2.518 | |
| 45,mlp.down_proj,0.0000061376,0.05000,8.347 | |
| 46,linear_attn.in_proj_qkv,0.0001213334,0.05000,2.041 | |
| 46,linear_attn.in_proj_z,0.0000596981,0.05000,1.663 | |
| 46,linear_attn.out_proj,0.0000044044,0.05000,2.046 | |
| 46,mlp.up_proj,0.0000939174,0.05000,2.470 | |
| 46,mlp.gate_proj,0.0000821531,0.05000,2.486 | |
| 46,mlp.down_proj,0.0000068771,0.05000,8.424 | |
| 47,self_attn.v_proj,0.0000223495,0.05000,3.554 | |
| 47,self_attn.k_proj,0.0000139771,0.05000,3.556 | |
| 47,self_attn.q_proj,0.0001322116,0.05000,3.625 | |
| 47,self_attn.o_proj,0.0000054021,0.05000,2.041 | |
| 47,mlp.gate_proj,0.0000914212,0.05000,2.445 | |
| 47,mlp.up_proj,0.0001044286,0.05000,2.472 | |
| 47,mlp.down_proj,0.0000083448,0.05000,8.296 | |
| 48,linear_attn.in_proj_qkv,0.0001325063,0.05000,1.871 | |
| 48,linear_attn.in_proj_z,0.0000630450,0.05000,1.650 | |
| 48,linear_attn.out_proj,0.0000050095,0.05000,2.173 | |
| 48,mlp.gate_proj,0.0000989087,0.05000,2.439 | |
| 48,mlp.up_proj,0.0001128583,0.05000,2.474 | |
| 48,mlp.down_proj,0.0000100373,0.05000,8.479 | |
| 49,linear_attn.in_proj_qkv,0.0001579284,0.05000,1.755 | |
| 49,linear_attn.in_proj_z,0.0000673421,0.05000,1.652 | |
| 49,linear_attn.out_proj,0.0000061752,0.05000,2.121 | |
| 49,mlp.gate_proj,0.0001063840,0.05000,2.492 | |
| 49,mlp.up_proj,0.0001208905,0.05000,2.494 | |
| 49,mlp.down_proj,0.0000127545,0.05000,8.585 | |
| 50,linear_attn.in_proj_qkv,0.0001606461,0.05000,1.742 | |
| 50,linear_attn.in_proj_z,0.0000741971,0.05000,1.635 | |
| 50,linear_attn.out_proj,0.0000095332,0.05000,2.041 | |
| 50,mlp.gate_proj,0.0001269221,0.05000,2.487 | |
| 50,mlp.up_proj,0.0001382575,0.05000,2.539 | |
| 50,mlp.down_proj,0.0000219604,0.05000,8.458 | |
| 51,self_attn.v_proj,0.0000416407,0.05000,3.567 | |
| 51,self_attn.k_proj,0.0000183768,0.05000,3.585 | |
| 51,self_attn.q_proj,0.0001753199,0.05000,3.645 | |
| 51,self_attn.o_proj,0.0000150616,0.05000,2.017 | |
| 51,mlp.gate_proj,0.0001431358,0.05000,2.525 | |
| 51,mlp.up_proj,0.0001607207,0.05000,2.566 | |
| 51,mlp.down_proj,0.0000273023,0.05000,8.495 | |
| 52,linear_attn.in_proj_qkv,0.0002451836,0.05000,1.755 | |
| 52,linear_attn.in_proj_z,0.0001126575,0.05000,1.738 | |
| 52,linear_attn.out_proj,0.0000120216,0.05000,2.091 | |
| 52,mlp.gate_proj,0.0001734989,0.05000,2.529 | |
| 52,mlp.up_proj,0.0001705315,0.05000,2.538 | |
| 52,mlp.down_proj,0.0000270397,0.05000,8.263 | |
| 53,linear_attn.in_proj_qkv,0.0002432394,0.05000,1.794 | |
| 53,linear_attn.in_proj_z,0.0001187352,0.05000,1.683 | |
| 53,linear_attn.out_proj,0.0000123070,0.05000,2.005 | |
| 53,mlp.up_proj,0.0001790341,0.05000,2.484 | |
| 53,mlp.gate_proj,0.0001916313,0.05000,2.572 | |
| 53,mlp.down_proj,0.0000296591,0.05000,8.487 | |
| 54,linear_attn.in_proj_qkv,0.0002351391,0.05000,1.690 | |
| 54,linear_attn.in_proj_z,0.0001260924,0.05000,1.679 | |
| 54,linear_attn.out_proj,0.0000132398,0.05000,2.064 | |
| 54,mlp.up_proj,0.0002111363,0.05000,2.438 | |
| 54,mlp.gate_proj,0.0002394779,0.05000,2.470 | |
| 54,mlp.down_proj,0.0000408813,0.05000,8.675 | |
| 55,self_attn.v_proj,0.0000542261,0.05000,3.500 | |
| 55,self_attn.k_proj,0.0000249806,0.05000,3.556 | |
| 55,self_attn.q_proj,0.0002292513,0.05000,3.592 | |
| 55,self_attn.o_proj,0.0000153712,0.05000,2.096 | |
| 55,mlp.gate_proj,0.0002307660,0.05000,2.567 | |
| 55,mlp.up_proj,0.0002120899,0.05000,2.586 | |
| 55,mlp.down_proj,0.0000343705,0.05000,8.457 | |
| 56,linear_attn.in_proj_qkv,0.0002710900,0.05000,1.717 | |
| 56,linear_attn.in_proj_z,0.0001354000,0.05000,1.654 | |
| 56,linear_attn.out_proj,0.0000174018,0.05000,2.036 | |
| 56,mlp.gate_proj,0.0002556430,0.05000,2.371 | |
| 56,mlp.up_proj,0.0002280795,0.05000,2.494 | |
| 56,mlp.down_proj,0.0000330924,0.05000,8.658 | |
| 57,linear_attn.in_proj_qkv,0.0002886057,0.05000,1.746 | |
| 57,linear_attn.in_proj_z,0.0001446880,0.05000,1.738 | |
| 57,linear_attn.out_proj,0.0000142626,0.05000,2.070 | |
| 57,mlp.gate_proj,0.0003000898,0.05000,2.449 | |
| 57,mlp.up_proj,0.0002629923,0.05000,2.490 | |
| 57,mlp.down_proj,0.0000357148,0.05000,8.592 | |
| 58,linear_attn.in_proj_qkv,0.0002797239,0.05000,1.755 | |
| 58,linear_attn.in_proj_z,0.0001519623,0.05000,1.643 | |
| 58,linear_attn.out_proj,0.0000166954,0.05000,2.101 | |
| 58,mlp.up_proj,0.0003074341,0.05000,2.466 | |
| 58,mlp.gate_proj,0.0003506556,0.05000,2.499 | |
| 58,mlp.down_proj,0.0000491605,0.05000,8.299 | |
| 59,self_attn.v_proj,0.0001172391,0.05000,3.485 | |
| 59,self_attn.k_proj,0.0000323962,0.05000,3.556 | |
| 59,self_attn.q_proj,0.0002820901,0.05000,3.629 | |
| 59,self_attn.o_proj,0.0000436953,0.05000,2.011 | |
| 59,mlp.up_proj,0.0003165537,0.05000,2.474 | |
| 59,mlp.gate_proj,0.0003521640,0.05000,2.493 | |
| 59,mlp.down_proj,0.0000602049,0.05000,8.295 | |
| 60,linear_attn.in_proj_qkv,0.0003418254,0.05000,1.732 | |
| 60,linear_attn.in_proj_z,0.0001630311,0.05000,1.743 | |
| 60,linear_attn.out_proj,0.0000372611,0.05000,2.074 | |
| 60,mlp.gate_proj,0.0003468314,0.05000,2.334 | |
| 60,mlp.up_proj,0.0003180261,0.05000,2.707 | |
| 60,mlp.down_proj,0.0000706684,0.05000,8.564 | |
| 61,linear_attn.in_proj_qkv,0.0002525166,0.05000,1.726 | |
| 61,linear_attn.in_proj_z,0.0001450246,0.05000,1.632 | |
| 61,linear_attn.out_proj,0.0000328215,0.05000,2.030 | |
| 61,mlp.up_proj,0.0003388942,0.05000,2.515 | |
| 61,mlp.gate_proj,0.0003665309,0.05000,2.535 | |
| 61,mlp.down_proj,0.0000886933,0.05000,8.448 | |
| 62,linear_attn.in_proj_qkv,0.0002738863,0.05000,1.847 | |
| 62,linear_attn.in_proj_z,0.0001453292,0.05000,1.619 | |
| 62,linear_attn.out_proj,0.0000932844,0.05000,2.057 | |
| 62,mlp.up_proj,0.0003140498,0.05000,2.411 | |
| 62,mlp.gate_proj,0.0003378174,0.05000,2.476 | |
| 62,mlp.down_proj,0.0001379779,0.05000,8.306 | |
| 63,self_attn.v_proj,0.0000941947,0.05000,3.473 | |
| 63,self_attn.k_proj,0.0000303032,0.05000,3.609 | |
| 63,self_attn.q_proj,0.0002578836,0.05000,3.627 | |
| 63,self_attn.o_proj,0.0000988559,0.05000,2.215 | |
| 63,mlp.gate_proj,0.0002963812,0.05000,2.448 | |
| 63,mlp.up_proj,0.0002661912,0.05000,2.565 | |
| 63,mlp.down_proj,0.0003060376,0.05000,8.574 | |