Instructions to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code")
model = AutoModelForCausalLM.from_pretrained("richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code

SGLang

How to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code with Docker Model Runner:
```
docker model run hf.co/richardyoung/Qwen2.5-Coder-7B-Instruct-heretic1x-code
```

Qwen2.5-Coder-7B-Instruct-heretic1x-code / tensor_hashes.json

richardyoung

Add files using upload-large-folder tool

25b23cf verified 4 days ago

Raw History Blame Contribute Delete

49.2 kB

{
"lm_head.weight": [
"torch.bfloat16",
[
152064,
3584
],
"57747b34119ec79b0ea5ee3404f0600ddd223526a5d9c7b102ef8ca719cfc06c"
],
"model.embed_tokens.weight": [
"torch.bfloat16",
[
152064,
3584
],
"6bab342c3d744197375af144a106d02a919848bfdaa787058fef29d66cff4ccd"
],
"model.layers.0.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"e981ef029184b603593435846827bb8eae0c98a00c2f1bb06a309907d5df6b77"
],
"model.layers.0.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"b3a5d8b04ccba914726b43b39ef8752c65624ec0d5b52ec2c7da2b1a3ee7b3cc"
],
"model.layers.0.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"cca3d0f3c626e32947222066b9e854ee069bc411cd8d820b1aa8324c7ec52da1"
],
"model.layers.0.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"420162f78361dd39bbf2f2dcec77bdc30818f1727cb1ff9f8334c713d0e915ba"
],
"model.layers.0.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"6afda044da0cdfed5269b4bd896b9783b7ef20adf134e19ddbbb151b336cb4dc"
],
"model.layers.0.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"9510ff69b652c12f637ca2f3ed8cbb8560551054333d2f4e6db135a00e1a49c4"
],
"model.layers.0.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"38b560551f9117f5b4f137e009e85134feb997ee4090f881934e9ca2a71c7cab"
],
"model.layers.0.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"0ebdefeb3f1891c93f82cd4483ae47c02ed7f8c51244bf9b34c1adef254eed3e"
],
"model.layers.0.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"9c7870a3861d67984a41cc2e79004f2a9d76276ac49e92b15731c689375b3ebb"
],
"model.layers.0.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"cff2f49a4e9229779f822dc0d050225ca30d6b9c963042d1c959e50a51515932"
],
"model.layers.0.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"5b5127b868273228dbe0413eda1196a931801b4150894c3066f0ad07c6efe340"
],
"model.layers.0.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"fa3fe35d33c1dcfed7123fd55bfb04f6a984bdef610e407d7a5a4b90e9997090"
],
"model.layers.1.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"8d96bc783650e953ed851f90be7373f11545adceb8f94c56d9432a4b27aeb6f1"
],
"model.layers.1.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"058d1d43dbb616c6e38bdf4b5909d694a3d8c06d67ef3a70425e9423de8cf838"
],
"model.layers.1.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"8cd30aef544cd6c832aafabe2254cc709c1467138110f194f331d8ff23b5f079"
],
"model.layers.1.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"426ac035e4daa5a6a20870a9db43f25d08626292bcb243a3985cdba6e2f87c5b"
],
"model.layers.1.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"f859d8b30938f8f0f2050ae90306e989ea661e5718cdb0275a9f2fea27b27bd5"
],
"model.layers.1.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"823292de5759d05e47669e247df7cfcb2320c78d88a3fa18215cef7f1b65b4e4"
],
"model.layers.1.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"bc694573f932fd729fe0ca93cfb2f34b198f4ddc155a47e8976922a26b0ccf01"
],
"model.layers.1.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"cad0811359f95d10164bf65dbff391066c85bdcdcb11f2cfcee6e0ea0c1b27d4"
],
"model.layers.1.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"88b15d04973c4c8b8bd1b61c9d1fbfff0ce2478ed0f5ec2cb9bb575e43a299e2"
],
"model.layers.1.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"6e262fbd51859ac1822e248dba5de7c6e4146b6b0fe5168842bf86b31a80556f"
],
"model.layers.1.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"87253c82f07f28627cf72389940a17cd9192d881a229d41c517a3ed0607fbc34"
],
"model.layers.1.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"24db9ecec06ef5bbc60bb527a897c0bcd61a72018ede00e20e46d777119cb6a3"
],
"model.layers.2.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"022d6bc00fcf25067a59ce8997a0ec6bb9eb6c0baf946faef04604c5092fdbb1"
],
"model.layers.2.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"666612c5e5564871322c44d34d7e90c445d694ce8e4177d255fb36a5fed2829d"
],
"model.layers.2.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"1a005b00fb64b6b6b38113f32a1f6278ffc6e2e1a88ff1eab13bbf58ad2ce5e3"
],
"model.layers.2.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"66c5b3a57c785332a7f32a9421363c2af9bda30460d82ba7a67c2c585330f067"
],
"model.layers.2.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"f8b626a85e99e91a88e7de8f44cb84a56af3c445f47e56da5c6c7d683664b599"
],
"model.layers.2.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"f077059faf03f2bc05a6f9350612b432a9d93e206d6f9abba78d027e1db2d2c6"
],
"model.layers.2.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"3d8eb3286631bfaafae127ac5409685daf1c6becaa41ec22c65cd92e5d15980c"
],
"model.layers.2.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"252983fd3ce6f40fd313aa6cc7c7723f1ab0a24062fc251c41c0db87df579faf"
],
"model.layers.2.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"846d9453007d2929792f6d6e3167d54014e8ee4871bd03716b7360c7196b3e5c"
],
"model.layers.2.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"4bffcf731da596375572e2b3eef0a579d79fe8b9c8a8f72a22585f8c1ed86b8f"
],
"model.layers.2.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"31fbd540be42a5ee8713fc6013c9ad8404be74b0aa6efb3303d7c0b0fdc53cfe"
],
"model.layers.2.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"23c3c48fb4d49e9d4f5c3ed0c2fab24b34b46d69a1df349af8754c2f22a11649"
],
"model.layers.3.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"d4e6d8f8d068714b049ceea13302a4d8c007f6d0eec5f98c95fe95bd18df6a85"
],
"model.layers.3.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"e7cfd4e18782e8a9c3e6ea05866cbd0e6fc73c038772735cea2cf196231b198c"
],
"model.layers.3.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"6f6b74958a8481566ca15ea0fbf8341de6aa9a1fed8133f45fd992d87c7ca299"
],
"model.layers.3.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"c72ca9ca85a3e6d543e06f4dd86480f24b6051e487caa81abdbc1b97cc7b0105"
],
"model.layers.3.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"fafb401bbba25ea443098d4415fc14837e7719acc8a4e2039999495aed9dcb46"
],
"model.layers.3.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"3d8ca6df6c9ddae258a98fd89eedc4c116bbe3a1e2c10e6a9130454813e5b919"
],
"model.layers.3.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"86b42f327bf494442bc3873f7c75caac4c006159e5bbebad9ee8c61280294065"
],
"model.layers.3.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"f903b3c54b53097355db1ab5fa7a6ccfa6908ce1333b4801bea663b7f661e002"
],
"model.layers.3.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"63153e51485e153ae6ea2d5f4abe241aeb38246137f44fd1ff6b8ac0ad56e0b4"
],
"model.layers.3.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"6dc65a13734c26db73dcfcb5b09c1bfdcd5fbad94c9865ca7dc6184d421a11c6"
],
"model.layers.3.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"fe4b8643965a1476600fc74a5334b2391caae30edb9fa93770b0f667bdc3905c"
],
"model.layers.3.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"6c50ef0132be0dd8179dfdd9d8288104b9e5359c0bf611283be5aab3fc483134"
],
"model.layers.4.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"af026bab7676800c369a4961d0dbeb4e1261fc655ebaded682c417e67737d5c8"
],
"model.layers.4.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"241da86bcea9441e3a39cf3dc1df5b09a6aa9947de7b6bdb5a9fe22d1dce4877"
],
"model.layers.4.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"ba48084196287f9ec7615ff2c143107b5bba8f6823869bdcc76cea1223541639"
],
"model.layers.4.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"a94697f57000ae1af6979b571773f09198ec69589f0fe12515a77ecb12530b66"
],
"model.layers.4.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"dc6adacbab7df990a20d36e4fe11734904d4f847aea9761b3f300e9e66ed4bde"
],
"model.layers.4.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"429cee15896bc4dafa3a3d9d0b325f2eeb776c35a462afaa35f8bd69da7dbe47"
],
"model.layers.4.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"0c8be4b8d945b0b7b962cd66309c98c302f6c904cd0e06106a03ef2f7c4844f7"
],
"model.layers.4.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"86fe5c4c88d061630c2ff2e8c69d0009959c9780efbf95ecfccf774d58fdce00"
],
"model.layers.4.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"19b93f6275cf067c80d000d3426d6c1931ee7ea55151b3d1be0e670560266b4f"
],
"model.layers.4.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"58ac1b5a87d4b013f32df45b1b27e0afc6916d2e8dddb9cfdd376910829f8824"
],
"model.layers.4.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"13b7f6566f5407be0e6ec0cb54e1198ab17dd9e81c8b16cc5f9291df9980a964"
],
"model.layers.4.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"d3c8205ee60b5bc9888ed3153126c1c798846682fd06401916e054e18c388349"
],
"model.layers.5.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"d4a05f9bc97c468aaf5b3fdbbc58a4de6f158e74f7aafd708742dc35f99482c3"
],
"model.layers.5.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"ef97ac499bdaf19813649d8d54742f7c160ae9ce699055913b4111b710ab4d05"
],
"model.layers.5.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"97fe22af6d273d93771f0557658f0e2c303108449bbde05787eb7f1478b7064f"
],
"model.layers.5.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"5be4138920db4489cf323dfc921cd33f2338a2ad574f7498a60fa009d29a0a8d"
],
"model.layers.5.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"c6560826840d525114dcfe24f93dbe199e2b4bec0715a8f6e414e2c9128c0e49"
],
"model.layers.5.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"c91205133026d4908f10d2b4c6def404b41e782d468d8434b5f234d9d79c9be2"
],
"model.layers.5.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"722fe45e487bc195ad000828b75b954c0bc963f2d388e9f1256da322a8f7908f"
],
"model.layers.5.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"a05e75e7b743f3b4ed09737186478ab82350812a522382d0ac2b4ad85549bdc4"
],
"model.layers.5.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"3ca2179b8add69d4e7edf15fc035a840d5598454d10830343122720eee6eab4f"
],
"model.layers.5.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"d2bd18987d601bacac0f709c592a86d7631d9635c04e060b27a47e6b70dc35ed"
],
"model.layers.5.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"1fe045ea78c5caa419ba69c7cf13551480b454cf6b7415e1e7247043336eede9"
],
"model.layers.5.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"5489546cb1ca0a387e7314e1d6ec2f609fbef5a4af3f67e48205d4c99fb8fb35"
],
"model.layers.6.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"20b63a0d43392480c490e31dfed48259cc76e44b9cbb146152bb37f7a7f23ba9"
],
"model.layers.10.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"7605ab633086f29c89afaa4c27d5daf7a730108593c890b275de110ddb3d91f8"
],
"model.layers.10.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"5aa3a000badb41ce4a45338719ab689477792d3ab1ad316426fb1bdd522cd07e"
],
"model.layers.10.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"87852f64d184a3c6922a04e968df1496fe975bf3a993190e1c0985a1fca13182"
],
"model.layers.10.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"9a571bc50f01adbb1dbe4c381a80180525fa9543f846e086a75c0b1420e71f79"
],
"model.layers.10.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"f6c03538c134238e3794cd9b1249b9484528335a594a643dd48b4dcfe1bd0f5a"
],
"model.layers.10.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"d19185798f9c4ee356f73ad41fe4a213e72c8f4797e656a8c7b7c1c19d27cc53"
],
"model.layers.10.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"c9bb240a706a5497d473390429c22ca5c8cf8aabf56e60d4e25867dee22a5876"
],
"model.layers.10.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"bc5843ceb90f018d02833724ddd39ebeb74693e279274dafe9b8b324b80e7bc5"
],
"model.layers.10.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"18c3c1dda1761fea905e18b99efb394d7c4fcb041790bc1969aab9ec9903eb98"
],
"model.layers.10.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"7d212d9164b2d27d4062e6eccdb71a98ddeba6fa3135a83f9274a57ae6556dde"
],
"model.layers.10.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"6e854f3b2d1f7e3b26e0f5d048f3df41ec883b6706070b81a3e352307fbd9594"
],
"model.layers.10.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"46d40c91008198da9a0a4d6a83dd38e3d96e0cd053d10a3804888ac8634740a8"
],
"model.layers.11.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"bf4d837af9595074e64bd9409e69c7c4f9460b661d6a019fb3ae12ab5ee844a9"
],
"model.layers.11.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"544f05a669a7197d06b220c705dd2a039a5029cf957732231c1bc6382baf8379"
],
"model.layers.11.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"d0905662d860dbfe7ae04d26299872d130df3ac1b03bb2a1a902c602d59d0ed6"
],
"model.layers.11.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"b5831f96b9fbfeceb6f8452d8248db2ad038d7c034f9527a2129952f2fc05b64"
],
"model.layers.11.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"6d81cee169558861ec4b53b386105242d97733aad7622e5ed3d5ca6b2183a1ab"
],
"model.layers.11.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"d874bfaeda01c2204ab299cf5c4bce6568118eaa13f3015ed1da487d031c7659"
],
"model.layers.11.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"6c5a28eb224d80e2c38fdb0ea36e9b520a8840868c4249966d9b4e9d6b2468f0"
],
"model.layers.11.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"8185bc38b2bd14bf188759ef7d6949fda06037149f4a472cf1001a8c0b9a0284"
],
"model.layers.11.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"c5d6b10d0b87ee460ef0be0c42be752542e3d81ed64dd213edf1224034147fc9"
],
"model.layers.11.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"01f70d9b308495b0fa9c6afb92212ea9c0b06c36305f29b0f0bfe1bee5339e3e"
],
"model.layers.11.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"260f1efc14fd44bd9272217404bf92aad242b7b2f6ffa67e003a53c7af9ff234"
],
"model.layers.11.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"bc0416a18706db8c0be8faf85ad15c845e35e124dabd4703d30e9adfee505e27"
],
"model.layers.12.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"ecfc4b78a521ab3850d2352820088f06a70ad8eed578e722b0eeebe94a883e4b"
],
"model.layers.12.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"f2297df6d4cfaa66c5f96ff2c309437e13828d227bd7952f400ceefa031b48d9"
],
"model.layers.12.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"9651fcfef4313d1bb259ca654db4471b0d02d029f809361de3d67067bb1b1707"
],
"model.layers.12.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"507ec705f92bf1d60c411ac91fa91d1d76122b1f175d161326de69395dde587b"
],
"model.layers.12.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"c47dc7ed289bfbc489378cd67ac681c19195e72d80c2b05eb891f2b2a71ff744"
],
"model.layers.12.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"c994d0f769b348bd3d3d4b6f0b37d7b98aa2c8f8309a2c0d280b8981fe1eb3df"
],
"model.layers.12.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"21a26628a25a7954c713ed00747c7065da3fc07914b1f7026e3b649ef2ae6148"
],
"model.layers.12.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"911b702af444c901c54f1dad42df40542eba4fb2fb1d4578df0b567542090eb4"
],
"model.layers.12.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"74090a9d318cd0b577eb999ffd313760fd5e103185d6892ed4958c8ebaa567f0"
],
"model.layers.12.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"fff887eca1e9d4b7b72e59a2005e53ea76ee0257032ff24a04b155f63ec34ca9"
],
"model.layers.12.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"7c2f4cb5b2cd85043fab49470dc31f5a30fb04cf31d2ad23fcbbc115f5c40466"
],
"model.layers.12.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"f214313171e41d563e3268591349f7493955e7992829838c944cbb4019360c8d"
],
"model.layers.13.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"a869472e9520f10ee3719d99131fecb1e27713d663c492f84865d901036312c7"
],
"model.layers.13.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"7c73e246832dafa48025a24a66a7b8dd9c606c840a6b2be3c77ab78cc1cb0f91"
],
"model.layers.13.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"fd68afc7bb3925b066f3297562180656e2aa15d0101babe6cc207806266e4093"
],
"model.layers.13.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"2f847dc37f7942fb8106300e26bfdf860ea2c39585f51833af2f15039c62c9dd"
],
"model.layers.13.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"fe630cab425d7b8732226d261b509fa157007b8c9216a17c57e3adbbe10c314c"
],
"model.layers.13.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"0ba1ace7ee8242c25375b19c4c39787880e06b0de7a9032d1a9fe3335c1d9a52"
],
"model.layers.13.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"1884734f9d1bca496fb244af0d9e7aac8b17065def342208fe2539575cf93d19"
],
"model.layers.13.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"6f589388afa599100ea803627d2b26b944c9017a22ae82a568d8186c46f6d324"
],
"model.layers.13.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"ab507e9e8390e281a1ebb432006a6e88ea5347a249c1558ab0725d3d5884b024"
],
"model.layers.13.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"29c19f7f76256c15f5f43b0e48d58e064d134c1a09a3799376c3aa8680a74b81"
],
"model.layers.13.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"928461c5351d45e27d62206c2f2ff5c33f09a02a2aaadc2fbcfc8deedb10c83f"
],
"model.layers.13.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"cad5692e3ce1943f25493f49c55db4f7a39c4f5721a813611a47bdca4b1d2900"
],
"model.layers.14.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"cad49114665e9de8cd6a1b01a84557ba1c86e8d73696b7a9fbbfefcee63e1a6c"
],
"model.layers.14.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"88052c96dc5450ec60049c69090401c84137bb2ec80461afba101176a7df1d33"
],
"model.layers.14.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"4819afc3dae40682871869d68338aec7fdd3f4fa77c7e97867f1fc27329f5d73"
],
"model.layers.14.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"211745ca8a6534904112fda4897b8e90cff1535876cd1ebe2ac5e0e510868b94"
],
"model.layers.14.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"7e47626a8818069aad7d220d50a0f7f23e8ce62e9a9972537952946b15a977f7"
],
"model.layers.14.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"dfde9f389e4216bb22475efc6db6b5ae4263bc61c74902428002206c990b50b8"
],
"model.layers.14.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"4ff389a69f35a1f793d926b6e33bf876385fb0276beaf5ba834762d0d4f7cb27"
],
"model.layers.14.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"20ba98ccaf8d8070c6707f91ca94b0fad1c0b1695a1443bf8d1bbbfe9dc91e43"
],
"model.layers.14.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"d42596ce91026ac927ae6f36445561d1197e415bff4f5508b96d9b401b39ec49"
],
"model.layers.14.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"995102d4f7e556b1560c6c5794f3ba69dfea14ccdaee7f6d311fe736697b2f6f"
],
"model.layers.14.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"adcba50bfe4bb754ba6f3e80da979dc6958b68fddb985d12f77bc79ebac190bb"
],
"model.layers.14.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"80e8230848d7c34be6aae04910486990b96303daca2afcb9d38f1ffaadecd8a3"
],
"model.layers.15.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"a7e62cc01e2a191fae523de9433db769098fb18bc3fd6b482a88a63cbc9c89e6"
],
"model.layers.15.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"ac01ffcfa3f2e5c1db437c37b776cbc51fc62bbb928cbb7b1eb4d44bb332d2e4"
],
"model.layers.15.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"f0ec1e6e12ffe996108ed2869e93d41b8451d2d480929c844319aa2137b899d4"
],
"model.layers.15.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"679cc8dbe9111d0618288d1a722f479b97a36232b8192f23b8a0181bdaa3e433"
],
"model.layers.15.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"2ea626a9ba83cb14ece9164ac52643d006ce603a4780ad91cc823cf45770ca6b"
],
"model.layers.15.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"cd5372fcc0450e5e531061e50fdf19438192e583d376aead5b3fbdac6a3a198f"
],
"model.layers.15.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"181927f0692e6d0b5b824948c4c3970bcb916eec46936fcba290b1805ce718df"
],
"model.layers.15.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"615d625571dacb03b61fe95e509222d53430510f567cc1b35ddd4c87c53bd5f7"
],
"model.layers.15.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"21cfdb963ece7e607c7298babe6cc2ad513ea6e4a6bc8d05f7d29f2d6260e843"
],
"model.layers.15.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"25319415884a543dc6976ea9d20807cfa61c061f1e21af18680e3cc560eede88"
],
"model.layers.15.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"e57964c2f6b21fbee46663cfc732d0a002a35bb79153d001a9ea29bc30ea2f4e"
],
"model.layers.15.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"b694faae9fef3f2d586a56f1a06612c200c1860ae50b942d230abf090bec7656"
],
"model.layers.16.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"35037e816fb9837576573230d09b47538e390d5000396dd00b276595879d518d"
],
"model.layers.16.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"42b676193aba177dba5c1ce1bfec96862c3e18fe9ef0be1d5b8bba40aefc80f3"
],
"model.layers.16.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"bec5ca11237d7dddbf6669c4a1fe2ebb959d0dcbdf144c12e827aee90af2233d"
],
"model.layers.6.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"ee09793dee528a68798b0f60942bdfd75a4caa8edff1cdf96c685126c33db410"
],
"model.layers.6.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"5550c9ba30502e0f7a8abcf5137460c8cf79b92fa6d24f9a96fe8d49e2575fdd"
],
"model.layers.6.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"cfd5d32991548abd8fac4a278cab1fdcbc54c24370f788193aaa500ff71525fc"
],
"model.layers.6.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"5a6a5671258fcd2c6a972fe25de02fe06d6d88a47ffe6de00d6c130995d5255e"
],
"model.layers.6.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"8432ecc898b135cfbf1cfcdab1c4002dd7c65e0bc13bd1bb9c72348fb31c9871"
],
"model.layers.6.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"45f9758eb6e206a3d68a1aefdc39b1c3a1d5f0dc6b8f9b3a1d0a7bb12a96b95d"
],
"model.layers.6.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"c74aa0185c00be49002e8fb14db8a0a972c0b3e581a63120d6bbd0b1f48803f3"
],
"model.layers.6.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"1edb60273f7d168dc2c605ddf3db8af66c7ea2ab83e2edb51a090c3b10c3532f"
],
"model.layers.6.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"8be5ea4a95f452ea450df072c3156038b0f2cf538fa0573f67cc75a4c3e72810"
],
"model.layers.6.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"b0984a8db73ef3cacc084972ccdf9a84f0a38fe77680fa34d886ef2de36c6c01"
],
"model.layers.6.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"67ce2f1e05543e0d2c911f8d290f5801db10167e45c28cb195d0cd599f7a9d43"
],
"model.layers.7.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"869e396985fce01834317254d443f4f9e619d80789c6e9219fcdaad78d9a89aa"
],
"model.layers.7.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"9dc9adcb64bf6fecf7248a609459145e2040fcca483c4954791e93ba638e00de"
],
"model.layers.7.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"098f1941c0e26335c296ff2b811e4949c810955efb6c8b4318afdbe8bdb03fee"
],
"model.layers.7.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"13c300a1ee5ae602026391e9347b47d0c6182dfc06806737fc8bbe5e7890855a"
],
"model.layers.7.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"c037c9d1bda58e050bf3972b474c85666b9c9cb8d013aac474047bbc6035aa29"
],
"model.layers.7.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"c22ec2e5e6c57e992aafeda9357f6b15d1dbd71a4b4ef156531d6adb58444a7c"
],
"model.layers.7.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"7abae83071c941e1bbdeaa0123dbcc6ede661da93cbd8ae42a30c2e6947d83e8"
],
"model.layers.7.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"b7ae5c1f13fcf844670363951037345f0d367c346d677eb1e1ec553894cc6a83"
],
"model.layers.7.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"5c04cbd72a8f66488df779ed7a825e00fbb1b0b7564c47f66cb602a209a471cc"
],
"model.layers.7.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"79b17d755c2f46dc3d803a2ed1b7f4b2385c9ed7f5a8a47145b86ed83574e5b2"
],
"model.layers.7.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"0107a169f9584c522790934ed2e506cacd50c5b04e0f1210712a3562142f3a40"
],
"model.layers.7.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"56a48ee782157cc61b716fd60d50030038ee0ed0796b262942688f17dde2a2fc"
],
"model.layers.8.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"ffb733c40c9d6bf504ab3774907760079b2fa66d75627ffa3be528151e49afad"
],
"model.layers.8.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"5234d43cb686642f6d5abf5d7a2ec191cd6e2493272e9a8441323536d57fa7ed"
],
"model.layers.8.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"0f0cb25e981f2800cce96abc246cb9ef285be263c3bcda872fe225024abf3df8"
],
"model.layers.8.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"40f52acf41c92bc3d9c711a0aaf0f44275d1323b4f38347710b91f45137de9fe"
],
"model.layers.8.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"c3f1986fe9194e54a6eace68ebc2be3619786c2f4bcc33a2045b45b23a761630"
],
"model.layers.8.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"ab5aef2605791eb1a7112a4b9bc87a3a6a9e0df3ddede01d1b2070cd05941be6"
],
"model.layers.8.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"e16bec36663d20e4cf21580dc8c206474e6610d74d846be54778fd5bfbba54ed"
],
"model.layers.8.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"92284b7b0e95000a3219d21334d5a276753d252369c388c4d5de9be498ff6961"
],
"model.layers.8.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"afd641e0216c93853f23c98346fc8a2fb295e00d532c7c94ce557c3143f53df0"
],
"model.layers.8.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"78b44848c4a2344ffd3bdc1a38376b34e34e332e100ac85191be7ce693c1f352"
],
"model.layers.8.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"e649e9acc08fb630bfc8457fd1985d9517e8386087200931ce31f97fadc6f83f"
],
"model.layers.8.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"3059c019ba571e312ac3361521a2d5af3b8d4943607cfaafd56135199660757b"
],
"model.layers.9.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"b58dbefabfaee9d44ad66745b1234132a72c795be8d65ad9479760739c0d8839"
],
"model.layers.9.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"823ca39bd303dfa6c8adc09923ade3f155db6cd3b5783386a1d54b6b008ff132"
],
"model.layers.9.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"dc4a4873f321de1aa77bacfdf4a33aa4d7c87eb684eff7cc188887de3fe741c5"
],
"model.layers.9.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"1cc270bf8c2ff51010acd78995da4ae41f453634e276f26506cb1444cbbb065c"
],
"model.layers.9.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"71495c62f164d62d155471f8eb01a3f0ad7491958d19f1d6b2e62d72f9b018cf"
],
"model.layers.9.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"b714813042676d6ef6af37021e5d0f1889acc563f57424c61123f3adbc06a22c"
],
"model.layers.9.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"522de6044cf58c0d55c03f7aaec25a8ceab618657c559d77ca37ffc34d98de9f"
],
"model.layers.9.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"1a03c257f8a1c4d9400691a8d6309f142bb186ceaac7edf0634af48f1a35cb10"
],
"model.layers.9.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"0ee8324ebfc63747dca6346eda3f3635eb952819d8d377a9ed8c04a0ddb04637"
],
"model.layers.9.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"2a955c55645f9d30f1c6f423690e05d22ccc4191ca4f0edc5983ce95e40cb996"
],
"model.layers.9.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"45d6b0f84a3c02693de7e432573270b4e0aac9f5feac22007bda815e7d9df0da"
],
"model.layers.9.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"ebb84436e1b3223da6e87f31511d5f106a26838b67852336bfcc3c7fcb91bd0d"
],
"model.layers.16.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"d3d0c504e68f76dff222a4650165ab2e54846b6691049c659d94f5d249086748"
],
"model.layers.16.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"1f032375b2c00dee0321e5cbe4ebe3a4601f8f26f3d37cf7987ed5d765bdeb88"
],
"model.layers.16.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"86b5ebadaefd42c93e96ce41c4ff18cdd4c97b70a1ba394d4b89265375f03dbb"
],
"model.layers.16.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"421f852c1101e9757a51ecb6fe6bd92215c173a8e6a6810f47d609376a152bc0"
],
"model.layers.16.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"28322fa1fe7d8dccf6ed99d04296465282fb04bd2607a01415909b41c84a1097"
],
"model.layers.16.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"d6d06eec139fec8be2c2dee0a53b0844dccbd466145e4bc47ed7c255b12ef962"
],
"model.layers.16.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"793ef44b626754f933a3d20603994537724b8d6482a750e4acb3f65c9663a50c"
],
"model.layers.16.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"3ee484f8de32d0cea10e9d19f332511194b507064f5b68cef3d714b6a32a3890"
],
"model.layers.16.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"dd2f805af82c2f5d3883b74a55d929d54dea4cff071f44c3e4f6dbcfe452a2fa"
],
"model.layers.17.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"6d479c211922e6894122712c88946378dbe5509e76e2b898508fb462ca4bc9af"
],
"model.layers.17.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"79eada983b6da2ca73b938e8b0587147585023d97ea4ac7cdcfc1b9f2db8adaa"
],
"model.layers.17.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"8ec33f07b0256e3748713bbdfe6d854197cce1562e576f7e1d7d0f78a6f93478"
],
"model.layers.17.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"ab8822e0cec7f40bb45fb0e00bb8622881c05f4d48adc4fbf9796802a9b07651"
],
"model.layers.17.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"8792890e4e357192d8bd94353f33b45da239ca04108f964951b2057eb320df67"
],
"model.layers.17.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"e5713c2141559e9f8d6029ef82ef1d68d9257663b15162a137541de59cea9ee9"
],
"model.layers.17.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"58eba3e23e771bcf070c007c7aebd1f7fff8b48d8f2db7f4c902ef09776d3f74"
],
"model.layers.17.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"ec19e168d1cb12036874118e5b1a5ea7986b161ff7aac59191e1bc0bcfba2fd1"
],
"model.layers.17.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"058165c22bae9ff13194f945fba40144894b3d0ed7fc4ef5727adb34297cd516"
],
"model.layers.17.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"5957b5576d5b4d19546b903f2d2662d24c9cc08c3027e56b7153c54d66739257"
],
"model.layers.17.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"cb01df3fec96ca0d99278ead839285a48d06b14a77e22fb3960aaf408435c0dc"
],
"model.layers.17.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"b99eeee079a58385dd93ea95e270d027648fc1b18e28cd935a441816c06522df"
],
"model.layers.18.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"ebea1185a767e77e1e3b7cc3e708e311840fc21fc54b8e13d99f145b92063e33"
],
"model.layers.18.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"42cc93e9a8e137fcddf324ed6f78bbc0dc077aa78b07ed59566ca0f81e1201dd"
],
"model.layers.18.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"af53f168fc91e8e7e94ee3b3261653b5f5ee64cb926981a4a555e4e519cde501"
],
"model.layers.18.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"a9349c715f8b268dcc7f64aa2aad17f26f7d866372f2bc1c7d5bcfd5312780d8"
],
"model.layers.18.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"c64a0f2667735450f31bdf34c9f4e2cddff83ac52f185c4ca627ad68dc865b99"
],
"model.layers.18.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"ed49729f0cfcd58bddeee635ef7dfe7e581543db51d24e7b2fb515021300a469"
],
"model.layers.18.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"0c309a8630b3d55ee7bf45cbe73d71a122f8164b11c44d876a66628240cba19b"
],
"model.layers.18.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"3d94629474fc5d3d0fd385738595e24710b2006bb798cd3d345aaee90fed458c"
],
"model.layers.18.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"42088427672da44a29a36cdcae16396edac71af7f7898580cfe155e111cb11c5"
],
"model.layers.18.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"10d8265cbe379099ab26b4705e31341470d19781cc4a21069752d8cbf8e583cc"
],
"model.layers.18.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"1bbbda30b919a41aadd1efb9ca9a4106d9b65132db52dbeaaafba23a862ae4d4"
],
"model.layers.18.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"dda4e97829578d4051c3f7c9a7c73b6b8bcb24b23ff1cb518c3f471960096b1c"
],
"model.layers.19.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"250a77d709f4c29a628d5301c13b70e91f2b9af4fa3dbda9e7cd78e9ffc555e5"
],
"model.layers.19.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"720d0daa65d03b574fc0c5144654bf91370b3e04ffea5b530d0f9c789228ae6e"
],
"model.layers.19.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"00cc0c9952de10614ca92317cb978f4ff8a5a95fc1907353cab1d0f5e35ea414"
],
"model.layers.19.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"0167f0a51ef186bb2ee77556769543d2d5fb4b96d43c298a7ee169c282648d7e"
],
"model.layers.19.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"258c0d921c568db3c5850251bd5a3b257194001c3179ff8a22211b61269d18e7"
],
"model.layers.19.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"0c87c94a6eb39fd70e9128a412ea61694d4d4a6be1a45f61f2ae0eaf8e53edef"
],
"model.layers.19.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"e1560be46b4a128b6d357895879b72f2f064808c86b68e380825fe22449d3baa"
],
"model.layers.19.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"0a64a53ffecaf1d876c7f05f0fd3cd507102d0a7547b0e958d3f44c326894868"
],
"model.layers.19.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"b08d8a27ea16693799baafbf7184d9bc1b748f309f76cffd6cdeb3283b7866df"
],
"model.layers.19.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"3bb9dc344c41f8c21e2efa2df88e6407d970dc7f6a0beb5a65c6862672b26ffe"
],
"model.layers.19.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"1df32c48f29abd322f0bc4bf4910bd4050f41f25d7e7d8e2b6c215b90f5561a0"
],
"model.layers.19.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"95d4f834d976fb8688e634fead81f77a370ecce05427e11ee70835b96e40df28"
],
"model.layers.20.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"4db017f0c10559c91fe352982ecfe9154ba3b183b6d505548e5cf7b5c4cd55b6"
],
"model.layers.20.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"ae45528c576b9fe814ebcbe21486fa77d6bd1af0520fef59f20d6487490d5a38"
],
"model.layers.20.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"019dee076fbd117cbd12376a5652a6288e0ff889bfea2483fbe33b09ef7e7436"
],
"model.layers.20.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"519842fd709cbe7b777b183c475b2ef82305c88b5244469013df28fac3a37476"
],
"model.layers.20.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"a6e16842641c3586aa751100863a22369d6016ba81073da208133e0ed8566fa0"
],
"model.layers.20.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"a5575dcc11b6da500a2dcd60d6607a1e20a6e043ffaad264d7196437e7d08c59"
],
"model.layers.20.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"59f0da13739043fca99b38f9cdce4faa0bc3711bb63617c1216b1d1a4fc8d4b8"
],
"model.layers.20.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"a08bab250bda42daebb1e23e07f24595dd0772cf85f494c68ba74cccdf43ebaf"
],
"model.layers.20.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"4fba7d3b93cacc7ea2fd9161929f3f0c86b540e787c5ebec4e566ee41e64b445"
],
"model.layers.20.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"08eb2ac2afe1e3b8736fab28963f55d4484a8bdb644fc3c0219181d02990d288"
],
"model.layers.20.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"4d67e0c23cb9669228035065756f90d19c71b9a36cf9b16e86db29901687eb38"
],
"model.layers.20.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"589af75345a4484b51a60ffa1d320b07de49c4f4822dd42e2e625cb084068fc7"
],
"model.layers.21.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"b1737aba69d394bfde1974509ae124012059364b950100ef40fe7b485414d6df"
],
"model.layers.21.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"9e8a70deab7fa9855cbf165b3da89d6db31fcfb41dcff0eb5c47cd67bcf20a3c"
],
"model.layers.21.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"e8323a83bdc9c0b77001ff05cc6a6e9484411128aa107d812b1f259bad9d8d9f"
],
"model.layers.21.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"c20e2cd7ca3c7ae4d02b79e5a1b6a7ddca191adf23b2a62fc1a4a23a5c2591b1"
],
"model.layers.21.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"9297b3fa3ab6b5046b0131b1461b8b12711f02e16e73216b99abfcfce49f3090"
],
"model.layers.21.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"cf962240acb65d2440834bffdc90cfa69b088c19b71679668a82ea132dc7f744"
],
"model.layers.21.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"ce0ac82e7a8cc28f6dab061272289c3882b424ac261800b271b8939a292e3e15"
],
"model.layers.21.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"98df1dcfdbbfc7f43636c6ddc7e2feccfa33302bb16b1a4f2251fa94ceb07b07"
],
"model.layers.21.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"cb77097436d72447d2aa577ec5020234e8f4e5f4952ec3ee17e39ed2dd527150"
],
"model.layers.21.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"621c3ec5c785faef5859d185e7481b7beb5e1edf1c3455558636dacc7d33d50f"
],
"model.layers.21.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"5dde760e91b2a13c44afd72aaf37cec4c9619449f9fac61b8cded457f95b896d"
],
"model.layers.21.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"dd5acc472b88af9fc5c58371e11d7992b1d34516097650367ab06105358fce66"
],
"model.layers.22.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"c9580e35a6bbe3c4e939cbc9c040806cacb14b39063e00204904f1fd771994e5"
],
"model.layers.22.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"36400383a1b5687d5b2dfdd7079bc0a2307746ea2669838987b9bc327417f46e"
],
"model.layers.22.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"fcfe8788c64ec31c2a819ea7916fca4e9e1b22ea6ba68fc80b1d5d14161c8677"
],
"model.layers.22.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"d4b4fa688b4acbabe8ef75415ec5b435bce72cc22ae668763ec6a62107dd240e"
],
"model.layers.22.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"b48fcb8c2095f64478b4ce8f8654013c2497335548475a5959bf9a4eaeb21ba1"
],
"model.layers.22.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"419a7c2ba33aa912cf6d1f5d19f0ff5d259ef10b7ef04727c960acfe7c1cc2c3"
],
"model.layers.22.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"948ba5673488665fe92fa2b906431954e569370633845ee0a517fda7377e1d0b"
],
"model.layers.22.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"18278f7d1826b1539cbed66bee002eec0bba3b7c0d5fac8a7999c14ddbbf647c"
],
"model.layers.22.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"7b3f82650ee6b4ab1e83c9fbdbbb1374018842b2f145f2a90e8e1fe6dcdb8105"
],
"model.layers.22.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"71bf4e1c191593a29323653dde916f8d100a4e94ff326a217015c4ba91d5a22e"
],
"model.layers.22.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"1246a8e4dc38abae33716f13912cf3f2741f477cc76f217985fcb6175504357d"
],
"model.layers.22.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"92376fe69341a102db600de44fa39a89cef55a95a9a215d091dda80323f6a61d"
],
"model.layers.23.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"7ccf1bbeb241618fd69277f8fc2a4d1c4c37516437896278135f7a03a7ddf1f2"
],
"model.layers.23.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"9deae973e4a881b01816d171d2f64b6a085d341dc54cd60627a678ee00bc01dd"
],
"model.layers.23.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"5e1e3eac2017a73e86cb3076709958daf7d848dc39a8a5fb8d326304e9654ed6"
],
"model.layers.23.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"882f2f892fc0310c17e2ba4dca90a22bfb1383fae8561e80d611204340c45b57"
],
"model.layers.23.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"6c1fbfb29ebaeabdd15d729a03f2d76bb4f3fdddd8198076fbf94c9183a1ae52"
],
"model.layers.23.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"113544c87ab16abe8eb3696ced1a3118535f451a36a6be50b382608f5d6da3a2"
],
"model.layers.23.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"6ad6a0f15d0bc70ecc52e8cce6132584d18ff48ab8948528cd19fed0e02a3a64"
],
"model.layers.23.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"76a957c1216a9283135211d662d9ff65b5617ec5c4c964a94f1b4e8903153353"
],
"model.layers.23.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"fe7ab39b7f4aa1b49d57466e74d96497bd61fbd05dbd10f6d81cd25ffeb45a80"
],
"model.layers.23.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"90b6d6c677951c4b768d8ebfb9dbe7153e963624899dcbb13434c516866f3b51"
],
"model.layers.23.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"10b1a23e14f5b3df8c6b111b4b5b151a59e04cf6e652a537d1b90995053dc125"
],
"model.layers.23.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"da5988c1545606b1216eebcc572cbc2f2441ce043a07abb01fd180fe5399219b"
],
"model.layers.24.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"815dd4dbbc2e327a807675f6f2f2ef3969b7bcfd632eaf296834e01e2f3f0b64"
],
"model.layers.24.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"c636cdedecc64def3202ff7038268ce10271877660b4ba06c95dd168a932303e"
],
"model.layers.24.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"7401f2970fe74ebdeae391f8e6b84d7967c3f3fe80cfc0e4403ea7a5bec24ec9"
],
"model.layers.24.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"5b74ff64cf2a50360314aabc1126a26849f0a8f7d275f4b1e5a4db371010fc9b"
],
"model.layers.24.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"6e118292cd5249b7617c86b9e433f2ce47e13f6d62766607a12dc9481a1fe78d"
],
"model.layers.24.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"99d391b659cd7cba9a3d7b90e56ae91c0e53356634ec48be4818c38414780a20"
],
"model.layers.24.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"fe37460f884b89fb33144d01e0cb91fcd57e01dd58a345d96901a5a19a0aa19b"
],
"model.layers.24.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"a42187ec124de262047806d27be24bb3e94e8fcd93ab9c7bd1d7712e7865a309"
],
"model.layers.24.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"85a1d738ad47d069151864163bf98e8adc87717a1cc0f17d3b91bc8f833bb14f"
],
"model.layers.24.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"19eecffa8ec50f577adcbacbdf294c3fba4777094a19ae6f3fa4b43223ad0922"
],
"model.layers.24.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"a5d56b564939a552ab3e52fc9dcc9ff44af4618e425b4c3ae23598d77683f355"
],
"model.layers.24.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"aa833779a510d64989b138c3f00665f8777514adb0025e6f669b829f3dcc46c3"
],
"model.layers.25.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"848ad605af964a2b6972d618460324b8f004b4f4d39c49e3809c516969deac71"
],
"model.layers.25.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"1b8c47c2b14d87acb178a73340ad621367f776d052b8793c9dfe737a6e242e49"
],
"model.layers.25.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"adc6bd7b796f07888ffa3a224dd5479bf180a935f5ab9afe6d77678f6c70cc4f"
],
"model.layers.25.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"a69cb6bbcd59b9307809be550d51efa4e99560afb5aca6ab1bfa07840cdf668a"
],
"model.layers.25.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"3ee3f1296c857bb034bc9f2882f2c2b007662d882c8d6a556f7c3fd924e02c33"
],
"model.layers.25.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"8fac628079dd41d092c5bd43057d85ccebc659741d46c3e4e2bfd6db134b233d"
],
"model.layers.25.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"5c33429a5db076f6d27caf94e5cf568d40a7c25a9d09c79e3cb4464abab4d10f"
],
"model.layers.25.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"854d6f69acde1d4972b16aa11f1e5ba1c160c3137aee0d24a21d7518b93a3c40"
],
"model.layers.25.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"7c7890b3f1a02da0088f49d9aee5bbf9f3cd1406df044283f4ede985883a57cd"
],
"model.layers.25.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"f5d0bd426740b50dafde3cd11fb4305e3a980f21ea7c22211e58bf3adffcc8a4"
],
"model.layers.25.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"b120af6065de82efb0b8ded7d1097de889ef392fca8ac19f850a9cb85acead8a"
],
"model.layers.25.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"e663836a7c12ed476c1823f8f1c62c72b3d3b25fe8bf95a6cbdd0c6412429fa1"
],
"model.layers.26.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"5278a388b622389ab9406c71a0635f18e4a79b0883463631a7670da8ec8a9a17"
],
"model.layers.26.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"95c942f30a7cc7601b5f3fc0633c46ebcba8f258d54796c9a319b96e5f976ab9"
],
"model.layers.26.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"9a6eef047b899acd9e5005245a2ba8e10470d297835497c95be208b90da3a3b3"
],
"model.layers.26.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"9f36f3dee6a247ff8bf1f25ba6e2656327b08ffa3c5e4310f10719ce7a58e6a5"
],
"model.layers.26.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"665da0ce6e7b18f6aa66cc12b42c22a1f20749827aa211bed89a6559c8d47e44"
],
"model.layers.26.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"328d297e95b8e018a87f6037adb1ea82b9aa702ae9f6a260d942ec6f4ce3cf3b"
],
"model.layers.26.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"44ad1cf7e7ebb2ed83b3196157692f3e02a108fed6b3f637b0a1c63ae9258c7f"
],
"model.layers.26.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"574069bbf52eaaf892f3731448c6975522227df324f86c4158a84907abe58f78"
],
"model.layers.26.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"046e2d1de3f6f9536a5cf1e844601a85de445cde3b45a9bde2286d747683dc84"
],
"model.layers.26.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"6ad17ba4f8dead7264fb15b4e95559ee4ba27f39cd161c1a57e156f15f397ee2"
],
"model.layers.26.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"5c127dd4762209f285e86380c3a48bc59fe44f92765f2a8092c8b13c53ef1704"
],
"model.layers.26.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"5b6ca96eb01f86afdc90a11f7e55d46b33f3d67de86a25bfa41706e448814b2a"
],
"model.layers.27.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"684f0f941ca94bf2f0c97e0a749f26b6ccb64b79e29e32a50a6b8b1c75dbaf92"
],
"model.layers.27.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"b62fae52083669ef6315fca2ff5eb92e8539e8bc082ec963c344006d62318c1e"
],
"model.layers.27.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"cf2154b1e090867a0d38492c9e91a484dcb7084a386ab71e3eac1ff72dbafca5"
],
"model.layers.27.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"821691f15139eb947c8f8c1d80a04696045d18f37ce3d6552e6b57eb3aa122b1"
],
"model.layers.27.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"a413ba5d800de72aef25a8146231f49a28cd25892de2ba8f46710b6984e54ac4"
],
"model.layers.27.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"d58dc9ab005949a54cf911032afb48e696faa81facec9f688c646bf6f5394855"
],
"model.layers.27.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"f512241a864c8ea24933a1eefa502249a17ada810058733a219f8a1a296bd355"
],
"model.layers.27.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"3e69650dc35c17d3ef5c0a81e67109f58a7fe3e3fde805d164ef1ba8da09c94d"
],
"model.layers.27.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"21b17eb41d544c2efb4f2b75851868d091a000d1d62f95bb5b9265f5edadfb5c"
],
"model.layers.27.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"f09a64aef6e1603839a32efa9664bbf77d9d87b6ff468fee157b9a08723223f2"
],
"model.layers.27.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"96f8ed04512741ea5063c2d50ef1eb5d58f4364606494cd3a927123d23cb8f05"
],
"model.layers.27.self_attn.v_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"d2c6a44d08712114e4c310e3e3596c028e7d90916c9d9ae6f879f574b9f8c4be"
],
"model.norm.weight": [
"torch.bfloat16",
[
3584
],
"1aa7aecd8620af69dfce219cdd32b5f0cf0f5a348ed0515bfef2529920293c9b"
]
}
{
"lm_head.weight": [
"torch.bfloat16",
[
152064,
3584
],
"57747b34119ec79b0ea5ee3404f0600ddd223526a5d9c7b102ef8ca719cfc06c"
],
"model.embed_tokens.weight": [
"torch.bfloat16",
[
152064,
3584
],
"6bab342c3d744197375af144a106d02a919848bfdaa787058fef29d66cff4ccd"
],
"model.layers.0.input_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"e981ef029184b603593435846827bb8eae0c98a00c2f1bb06a309907d5df6b77"
],
"model.layers.0.mlp.down_proj.weight": [
"torch.bfloat16",
[
3584,
18944
],
"b3a5d8b04ccba914726b43b39ef8752c65624ec0d5b52ec2c7da2b1a3ee7b3cc"
],
"model.layers.0.mlp.gate_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"cca3d0f3c626e32947222066b9e854ee069bc411cd8d820b1aa8324c7ec52da1"
],
"model.layers.0.mlp.up_proj.weight": [
"torch.bfloat16",
[
18944,
3584
],
"420162f78361dd39bbf2f2dcec77bdc30818f1727cb1ff9f8334c713d0e915ba"
],
"model.layers.0.post_attention_layernorm.weight": [
"torch.bfloat16",
[
3584
],
"6afda044da0cdfed5269b4bd896b9783b7ef20adf134e19ddbbb151b336cb4dc"
],
"model.layers.0.self_attn.k_proj.bias": [
"torch.bfloat16",
[
512
],
"9510ff69b652c12f637ca2f3ed8cbb8560551054333d2f4e6db135a00e1a49c4"
],
"model.layers.0.self_attn.k_proj.weight": [
"torch.bfloat16",
[
512,
3584
],
"38b560551f9117f5b4f137e009e85134feb997ee4090f881934e9ca2a71c7cab"
],
"model.layers.0.self_attn.o_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"0ebdefeb3f1891c93f82cd4483ae47c02ed7f8c51244bf9b34c1adef254eed3e"
],
"model.layers.0.self_attn.q_proj.bias": [
"torch.bfloat16",
[
3584
],
"9c7870a3861d67984a41cc2e79004f2a9d76276ac49e92b15731c689375b3ebb"
],
"model.layers.0.self_attn.q_proj.weight": [
"torch.bfloat16",
[
3584,
3584
],
"cff2f49a4e9229779f822dc0d050225ca30d6b9c963042d1c959e50a51515932"
],
"model.layers.0.self_attn.v_proj.bias": [
"torch.bfloat16",
[
512
],
"5b5127b868273228dbe0413eda1196a931801b4150894c3066f0ad07c6efe340"
],