Instructions to use Jeesup/svd-safety-l2_remove40_random_b005 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jeesup/svd-safety-l2_remove40_random_b005 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Jeesup/svd-safety-l2_remove40_random_b005")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Jeesup/svd-safety-l2_remove40_random_b005")
model = AutoModelForCausalLM.from_pretrained("Jeesup/svd-safety-l2_remove40_random_b005", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Jeesup/svd-safety-l2_remove40_random_b005 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jeesup/svd-safety-l2_remove40_random_b005"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeesup/svd-safety-l2_remove40_random_b005",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Jeesup/svd-safety-l2_remove40_random_b005

SGLang

How to use Jeesup/svd-safety-l2_remove40_random_b005 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jeesup/svd-safety-l2_remove40_random_b005" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeesup/svd-safety-l2_remove40_random_b005",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jeesup/svd-safety-l2_remove40_random_b005" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeesup/svd-safety-l2_remove40_random_b005",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Jeesup/svd-safety-l2_remove40_random_b005 with Docker Model Runner:
```
docker model run hf.co/Jeesup/svd-safety-l2_remove40_random_b005
```

svd-safety-l2_remove40_random_b005 / compression.json

Jeesup

random arm at 0.005 budget

29f62ac verified about 1 month ago

Raw History Blame Contribute Delete

48.1 kB

Invalid JSON:Unexpected token 'I', ..." "tau": Infinity, "... is not valid JSON

	{
	"source": "/home/users/jeesup/svd_safety/results/svdllm_full/llama2_7b_chat_remove_40_seed42/checkpoint",
	"method": "svdllm+rescue",
	"arm": "random",
	"budget_frac": 0.005,
	"ratio_removed": 0.3950200056164993,
	"param_fraction": 0.6049799943835007,
	"added_params": 33281024,
	"target_params": 33284096,
	"n_components": 3040,
	"n_swapped_out": 0,
	"selector_stats": {
	"orr_spent": -0.05065116121686905,
	"orr_capped": false,
	"pred_safe": 0.017037552411285264,
	"pred_benign": 0.0,
	"pred_dsafe": 0.017037552411285264,
	"pred_orr": -0.05065116121686905,
	"pred_refh": -0.010847308066169025,
	"pred_refb": -0.025702699877342125,
	"pred_gap": -0.014855391852460357
	},
	"mu": 3.0,
	"nu": 1.0,
	"tau": Infinity,
	"matrices_touched": 224,
	"exhausted": false,
	"scores": "/home/users/jeesup/svd_safety/results/recovery_jbb/sensitivity/remove40_disc_gap.npz",
	"basis": "/home/users/jeesup/svd_safety/assets/svdbasis/llama2_7b_chat_n256_s42",
	"per_matrix": {
	"29.o_proj": 17,
	"9.down_proj": 12,
	"2.o_proj": 13,
	"23.k_proj": 17,
	"28.k_proj": 13,
	"31.k_proj": 14,
	"14.q_proj": 19,
	"20.v_proj": 13,
	"29.up_proj": 6,
	"15.down_proj": 14,
	"1.gate_proj": 11,
	"22.gate_proj": 16,
	"12.o_proj": 17,
	"0.o_proj": 13,
	"10.v_proj": 17,
	"10.up_proj": 22,
	"20.o_proj": 17,
	"8.o_proj": 17,
	"29.gate_proj": 12,
	"14.o_proj": 10,
	"25.o_proj": 15,
	"17.gate_proj": 13,
	"30.q_proj": 12,
	"1.o_proj": 19,
	"24.down_proj": 14,
	"11.k_proj": 16,
	"6.k_proj": 23,
	"8.v_proj": 19,
	"3.up_proj": 14,
	"23.up_proj": 21,
	"4.k_proj": 19,
	"18.v_proj": 17,
	"0.q_proj": 16,
	"23.v_proj": 18,
	"1.down_proj": 19,
	"26.k_proj": 20,
	"5.o_proj": 16,
	"22.v_proj": 15,
	"26.o_proj": 11,
	"29.v_proj": 17,
	"3.v_proj": 12,
	"16.up_proj": 13,
	"25.q_proj": 18,
	"20.q_proj": 17,
	"18.o_proj": 8,
	"2.down_proj": 16,
	"7.down_proj": 11,
	"13.o_proj": 15,
	"27.up_proj": 15,
	"2.up_proj": 9,
	"12.down_proj": 14,
	"24.up_proj": 13,
	"24.k_proj": 11,
	"25.k_proj": 13,
	"6.up_proj": 19,
	"24.gate_proj": 20,
	"17.v_proj": 16,
	"21.o_proj": 11,
	"4.gate_proj": 15,
	"0.k_proj": 19,
	"26.up_proj": 13,
	"27.o_proj": 15,
	"21.q_proj": 16,
	"13.gate_proj": 11,
	"4.o_proj": 15,
	"9.o_proj": 15,
	"29.k_proj": 15,
	"7.up_proj": 8,
	"8.gate_proj": 8,
	"4.up_proj": 12,
	"30.gate_proj": 13,
	"5.v_proj": 12,
	"19.up_proj": 16,
	"14.up_proj": 15,
	"16.o_proj": 13,
	"29.down_proj": 20,
	"1.up_proj": 7,
	"22.up_proj": 15,
	"14.down_proj": 9,
	"13.v_proj": 19,
	"12.gate_proj": 11,
	"0.down_proj": 12,
	"9.k_proj": 9,
	"18.k_proj": 20,
	"24.v_proj": 12,
	"30.v_proj": 14,
	"23.q_proj": 13,
	"19.v_proj": 15,
	"13.up_proj": 9,
	"28.down_proj": 16,
	"5.k_proj": 18,
	"2.q_proj": 14,
	"17.up_proj": 19,
	"22.o_proj": 15,
	"17.q_proj": 14,
	"14.k_proj": 14,
	"25.down_proj": 11,
	"15.q_proj": 14,
	"12.k_proj": 15,
	"7.o_proj": 14,
	"5.up_proj": 10,
	"27.q_proj": 21,
	"9.q_proj": 12,
	"0.up_proj": 13,
	"4.down_proj": 15,
	"16.k_proj": 11,
	"30.up_proj": 8,
	"12.v_proj": 20,
	"23.gate_proj": 11,
	"22.q_proj": 15,
	"26.gate_proj": 10,
	"30.k_proj": 12,
	"18.gate_proj": 14,
	"8.up_proj": 16,
	"28.up_proj": 12,
	"17.o_proj": 13,
	"16.q_proj": 10,
	"5.gate_proj": 16,
	"23.down_proj": 13,
	"13.q_proj": 12,
	"20.down_proj": 14,
	"6.v_proj": 11,
	"6.q_proj": 15,
	"3.gate_proj": 16,
	"14.v_proj": 12,
	"15.v_proj": 10,
	"15.k_proj": 14,
	"20.k_proj": 10,
	"3.q_proj": 12,
	"11.o_proj": 12,
	"2.v_proj": 13,
	"22.down_proj": 14,
	"28.v_proj": 19,
	"7.q_proj": 16,
	"18.q_proj": 17,
	"10.o_proj": 22,
	"31.gate_proj": 10,
	"21.up_proj": 16,
	"10.k_proj": 13,
	"28.q_proj": 15,
	"4.v_proj": 9,
	"18.down_proj": 12,
	"12.q_proj": 14,
	"31.o_proj": 15,
	"30.down_proj": 16,
	"1.k_proj": 9,
	"27.k_proj": 13,
	"25.gate_proj": 8,
	"9.up_proj": 11,
	"31.q_proj": 13,
	"17.k_proj": 15,
	"23.o_proj": 11,
	"9.v_proj": 10,
	"6.gate_proj": 8,
	"0.v_proj": 17,
	"24.q_proj": 11,
	"8.q_proj": 19,
	"31.down_proj": 11,
	"9.gate_proj": 13,
	"21.down_proj": 10,
	"16.gate_proj": 13,
	"26.down_proj": 16,
	"6.down_proj": 16,
	"13.down_proj": 6,
	"10.q_proj": 15,
	"30.o_proj": 14,
	"15.up_proj": 16,
	"5.down_proj": 5,
	"27.down_proj": 14,
	"15.o_proj": 15,
	"19.q_proj": 22,
	"26.v_proj": 8,
	"0.gate_proj": 17,
	"1.q_proj": 11,
	"24.o_proj": 15,
	"25.v_proj": 14,
	"29.q_proj": 17,
	"7.v_proj": 11,
	"8.down_proj": 11,
	"28.gate_proj": 8,
	"16.v_proj": 18,
	"27.v_proj": 19,
	"19.gate_proj": 13,
	"4.q_proj": 9,
	"3.down_proj": 9,
	"20.up_proj": 14,
	"11.q_proj": 12,
	"11.gate_proj": 13,
	"8.k_proj": 12,
	"2.k_proj": 12,
	"14.gate_proj": 6,
	"3.k_proj": 12,
	"21.gate_proj": 10,
	"15.gate_proj": 10,
	"10.gate_proj": 16,
	"28.o_proj": 17,
	"12.up_proj": 16,
	"25.up_proj": 15,
	"17.down_proj": 14,
	"1.v_proj": 15,
	"18.up_proj": 18,
	"26.q_proj": 12,
	"11.down_proj": 13,
	"21.v_proj": 10,
	"31.up_proj": 12,
	"20.gate_proj": 15,
	"3.o_proj": 13,
	"11.v_proj": 16,
	"21.k_proj": 11,
	"19.down_proj": 8,
	"31.v_proj": 8,
	"19.o_proj": 12,
	"5.q_proj": 9,
	"19.k_proj": 11,
	"2.gate_proj": 12,
	"16.down_proj": 7,
	"7.k_proj": 12,
	"6.o_proj": 14,
	"22.k_proj": 13,
	"13.k_proj": 15,
	"10.down_proj": 12,
	"7.gate_proj": 4,
	"11.up_proj": 7,
	"27.gate_proj": 5
	},
	"components": {
	"29.o_proj": [
	1340,
	1345,
	1494,
	1736,
	1914,
	2063,
	2141,
	2180,
	2718,
	2901,
	3080,
	3444,
	3462,
	3471,
	3601,
	3707,
	4014
	],
	"9.down_proj": [
	2133,
	2392,
	2440,
	2530,
	2587,
	2659,
	2708,
	2931,
	3009,
	3490,
	3513,
	3626
	],
	"2.o_proj": [
	1263,
	1541,
	1644,
	1694,
	1941,
	1996,
	2201,
	2355,
	2879,
	2959,
	3130,
	3250,
	3493
	],
	"23.k_proj": [
	1253,
	1268,
	1535,
	1727,
	2077,
	2102,
	2559,
	2659,
	3049,
	3056,
	3152,
	3306,
	3351,
	3415,
	3513,
	3619,
	3797
	],
	"28.k_proj": [
	1568,
	1727,
	1734,
	1866,
	2088,
	2321,
	2594,
	3231,
	3281,
	3588,
	3847,
	3922,
	4095
	],
	"31.k_proj": [
	1371,
	1815,
	1868,
	2023,
	2082,
	2247,
	2462,
	2834,
	2867,
	3000,
	3009,
	3340,
	3396,
	3721
	],
	"14.q_proj": [
	1310,
	1374,
	1499,
	1586,
	1622,
	1774,
	2011,
	2126,
	2181,
	2354,
	2590,
	3053,
	3389,
	3426,
	3521,
	3770,
	3818,
	3854,
	3909
	],
	"20.v_proj": [
	1488,
	1558,
	1673,
	1725,
	1923,
	2416,
	2998,
	3169,
	3676,
	3725,
	3810,
	3827,
	3876
	],
	"29.up_proj": [
	1991,
	1999,
	2146,
	3138,
	3197,
	3528
	],
	"15.down_proj": [
	1881,
	1965,
	2263,
	2299,
	2347,
	2420,
	2684,
	2814,
	2889,
	2944,
	3051,
	3652,
	4033,
	4082
	],
	"1.gate_proj": [
	1844,
	1900,
	1914,
	1968,
	2079,
	2120,
	2257,
	2577,
	2629,
	2681,
	3424
	],
	"22.gate_proj": [
	1865,
	2138,
	2380,
	2401,
	2678,
	2691,
	2758,
	3008,
	3019,
	3594,
	3623,
	3624,
	3789,
	3840,
	4044,
	4085
	],
	"12.o_proj": [
	1324,
	1492,
	1693,
	2173,
	2542,
	2630,
	2860,
	3058,
	3084,
	3092,
	3294,
	3394,
	3516,
	3619,
	3669,
	3790,
	4046
	],
	"0.o_proj": [
	1241,
	1337,
	1350,
	1430,
	2807,
	2925,
	2976,
	3126,
	3232,
	3417,
	3773,
	3784,
	4063
	],
	"10.v_proj": [
	1547,
	2063,
	2286,
	2295,
	2336,
	2475,
	2597,
	2648,
	2704,
	2750,
	2839,
	2849,
	2853,
	3263,
	3479,
	3731,
	3848
	],
	"10.up_proj": [
	1868,
	1877,
	1964,
	1985,
	2123,
	2130,
	2144,
	2576,
	2578,
	2924,
	3033,
	3206,
	3504,
	3512,
	3654,
	3685,
	3768,
	3789,
	3873,
	3874,
	4033,
	4045
	],
	"20.o_proj": [
	1405,
	1450,
	1703,
	1880,
	1942,
	1974,
	2075,
	2259,
	2313,
	2492,
	2722,
	3008,
	3088,
	3118,
	3354,
	3372,
	3459
	],
	"8.o_proj": [
	1347,
	1462,
	1525,
	1614,
	1778,
	1964,
	2475,
	2582,
	2636,
	2703,
	2846,
	2906,
	3174,
	3271,
	3303,
	3506,
	3702
	],
	"29.gate_proj": [
	1809,
	1871,
	1933,
	2096,
	2164,
	2378,
	2720,
	3305,
	3380,
	3796,
	4012,
	4035
	],
	"14.o_proj": [
	1452,
	1748,
	2125,
	2192,
	2345,
	2479,
	2747,
	2981,
	3380,
	3877
	],
	"25.o_proj": [
	1282,
	1493,
	1716,
	1765,
	1866,
	2180,
	2648,
	2834,
	2931,
	3252,
	3363,
	3422,
	3902,
	3971,
	4028
	],
	"17.gate_proj": [
	1965,
	2005,
	2399,
	2766,
	2999,
	3099,
	3409,
	3490,
	3505,
	3918,
	4024,
	4071,
	4085
	],
	"30.q_proj": [
	1285,
	1529,
	1863,
	2099,
	2146,
	2736,
	2771,
	3397,
	3722,
	3828,
	3880,
	4037
	],
	"1.o_proj": [
	1239,
	1512,
	1621,
	1826,
	1961,
	1965,
	2068,
	2101,
	2162,
	2396,
	2595,
	2687,
	2806,
	2908,
	3025,
	3056,
	3153,
	3206,
	3642
	],
	"24.down_proj": [
	2182,
	2245,
	2257,
	2409,
	2936,
	3203,
	3232,
	3284,
	3570,
	3672,
	3824,
	3843,
	3854,
	4016
	],
	"11.k_proj": [
	1449,
	1477,
	1642,
	1708,
	1915,
	2297,
	2364,
	2760,
	2892,
	3030,
	3299,
	3427,
	3669,
	3781,
	3827,
	4013
	],
	"6.k_proj": [
	1273,
	1314,
	1560,
	1735,
	1801,
	1806,
	1929,
	1964,
	2055,
	2385,
	2437,
	3111,
	3189,
	3394,
	3419,
	3446,
	3528,
	3532,
	3563,
	3571,
	3600,
	3605,
	3670
	],
	"8.v_proj": [
	1354,
	1528,
	1560,
	1628,
	1726,
	2020,
	2093,
	2121,
	2601,
	3010,
	3012,
	3231,
	3348,
	3359,
	3441,
	3556,
	3640,
	3957,
	4012
	],
	"3.up_proj": [
	1856,
	2194,
	2398,
	2464,
	2470,
	2626,
	2912,
	3161,
	3349,
	3618,
	3787,
	4015,
	4054,
	4055
	],
	"23.up_proj": [
	1808,
	1857,
	1870,
	1964,
	2286,
	2302,
	2326,
	2353,
	2526,
	2581,
	2595,
	2742,
	2849,
	3134,
	3224,
	3416,
	3478,
	3648,
	3787,
	3993,
	4022
	],
	"4.k_proj": [
	1505,
	1841,
	2037,
	2294,
	2303,
	2328,
	2385,
	2478,
	2556,
	2864,
	3099,
	3127,
	3168,
	3304,
	3726,
	3876,
	3907,
	3914,
	3971
	],
	"18.v_proj": [
	1392,
	1639,
	1816,
	1887,
	1917,
	2256,
	2476,
	2646,
	2700,
	3177,
	3326,
	3432,
	3484,
	3602,
	3778,
	3798,
	3968
	],
	"0.q_proj": [
	1317,
	1607,
	1756,
	1869,
	1914,
	1967,
	2393,
	2695,
	2743,
	2819,
	2898,
	2957,
	3006,
	3178,
	3310,
	3721
	],
	"23.v_proj": [
	1584,
	1649,
	1904,
	2244,
	2309,
	2425,
	2437,
	2590,
	2667,
	2794,
	2827,
	2830,
	3540,
	3603,
	3624,
	3716,
	3774,
	3965
	],
	"1.down_proj": [
	1844,
	1987,
	2199,
	2343,
	2477,
	2733,
	2758,
	2970,
	2995,
	3006,
	3245,
	3334,
	3344,
	3405,
	3641,
	3702,
	3799,
	3953,
	4045
	],
	"26.k_proj": [
	1588,
	1637,
	1681,
	1901,
	1903,
	2021,
	2219,
	2580,
	2839,
	2927,
	2963,
	3195,
	3249,
	3346,
	3377,
	3416,
	3772,
	3924,
	3941,
	3961
	],
	"5.o_proj": [
	1437,
	1440,
	1682,
	1742,
	2010,
	2143,
	2212,
	2910,
	2984,
	3071,
	3321,
	3410,
	3616,
	3712,
	3785,
	3980
	],
	"22.v_proj": [
	1410,
	1440,
	1786,
	1895,
	1941,
	2017,
	2301,
	2315,
	2775,
	2805,
	2972,
	3058,
	3667,
	3740,
	3794
	],
	"26.o_proj": [
	1931,
	2221,
	2969,
	3244,
	3312,
	3329,
	3347,
	3355,
	3403,
	3824,
	3939
	],
	"29.v_proj": [
	1360,
	1468,
	1481,
	1546,
	1705,
	1970,
	2255,
	2452,
	2522,
	2530,
	3187,
	3227,
	3462,
	3624,
	3858,
	3916,
	4059
	],
	"3.v_proj": [
	1566,
	2002,
	2130,
	2455,
	2624,
	2675,
	2741,
	2852,
	3356,
	3419,
	3430,
	3554
	],
	"16.up_proj": [
	2090,
	2230,
	2455,
	2651,
	2686,
	2727,
	2850,
	2892,
	2940,
	3487,
	3580,
	3720,
	3826
	],
	"25.q_proj": [
	1518,
	1562,
	1877,
	2752,
	2833,
	2895,
	2920,
	3042,
	3203,
	3236,
	3335,
	3391,
	3581,
	3659,
	3745,
	3774,
	3825,
	4065
	],
	"20.q_proj": [
	1966,
	2391,
	2421,
	2504,
	2676,
	2680,
	2695,
	2728,
	2907,
	2954,
	3079,
	3495,
	3652,
	3683,
	3708,
	3795,
	3875
	],
	"18.o_proj": [
	1649,
	1659,
	1706,
	2922,
	3051,
	3320,
	3363,
	4076
	],
	"2.down_proj": [
	1793,
	2078,
	2163,
	2522,
	2753,
	2791,
	2891,
	2927,
	2961,
	3111,
	3253,
	3518,
	3526,
	3688,
	3788,
	3805
	],
	"7.down_proj": [
	2492,
	3117,
	3442,
	3466,
	3504,
	3737,
	3755,
	3838,
	3894,
	3992,
	4091
	],
	"13.o_proj": [
	1306,
	1800,
	1809,
	2032,
	2065,
	2140,
	2171,
	2459,
	2631,
	2664,
	2938,
	3410,
	3471,
	3472,
	3864
	],
	"27.up_proj": [
	1936,
	2006,
	2301,
	2684,
	2719,
	2728,
	2769,
	2936,
	2966,
	3115,
	3164,
	3361,
	3521,
	4025,
	4091
	],
	"2.up_proj": [
	1889,
	2014,
	2195,
	2469,
	2935,
	3514,
	3573,
	3715,
	3833
	],
	"12.down_proj": [
	2030,
	2064,
	2331,
	2681,
	2721,
	2734,
	2741,
	2833,
	3043,
	3131,
	3595,
	3758,
	3952,
	3992
	],
	"24.up_proj": [
	1965,
	2082,
	2502,
	2765,
	2867,
	3112,
	3381,
	3476,
	3619,
	3731,
	3749,
	3795,
	4089
	],
	"24.k_proj": [
	1435,
	1508,
	1621,
	1773,
	2348,
	2546,
	2781,
	2826,
	3403,
	3599,
	4032
	],
	"25.k_proj": [
	1559,
	1771,
	2370,
	2388,
	2584,
	2888,
	3041,
	3340,
	3395,
	3646,
	3718,
	3976,
	4057
	],
	"6.up_proj": [
	1798,
	1989,
	2008,
	2201,
	2281,
	2391,
	2457,
	2487,
	2567,
	2696,
	2764,
	3043,
	3092,
	3426,
	3445,
	3571,
	3605,
	3805,
	4045
	],
	"24.gate_proj": [
	1817,
	1869,
	1878,
	1983,
	2326,
	2430,
	2436,
	2604,
	2663,
	2787,
	2796,
	2906,
	2917,
	3169,
	3477,
	3532,
	3606,
	3696,
	4006,
	4035
	],
	"17.v_proj": [
	1266,
	1380,
	1399,
	1428,
	1466,
	1513,
	1584,
	1724,
	2010,
	2232,
	2434,
	2854,
	2889,
	3235,
	3329,
	3453
	],
	"21.o_proj": [
	1266,
	1822,
	1986,
	2378,
	2523,
	3139,
	3148,
	3284,
	3655,
	3921,
	3954
	],
	"4.gate_proj": [
	1797,
	2035,
	2122,
	2212,
	2224,
	2636,
	2731,
	3339,
	3414,
	3593,
	3789,
	3790,
	3836,
	3868,
	4068
	],
	"0.k_proj": [
	1302,
	1399,
	1507,
	1663,
	1712,
	1924,
	1989,
	2097,
	2335,
	2398,
	2468,
	2591,
	2661,
	3143,
	3212,
	3468,
	3743,
	3770,
	4016
	],
	"26.up_proj": [
	1836,
	2037,
	2116,
	2244,
	2360,
	2816,
	2881,
	3089,
	3621,
	3707,
	3722,
	3900,
	3963
	],
	"27.o_proj": [
	1253,
	1256,
	1264,
	1393,
	1928,
	2309,
	2570,
	2748,
	2984,
	3076,
	3179,
	3332,
	3446,
	3489,
	4035
	],
	"21.q_proj": [
	1314,
	1456,
	1596,
	1736,
	1790,
	2149,
	2328,
	2714,
	2791,
	3021,
	3378,
	3466,
	3669,
	3725,
	3802,
	3853
	],
	"13.gate_proj": [
	1929,
	1954,
	1968,
	2076,
	2476,
	3244,
	3317,
	3484,
	3589,
	3658,
	3990
	],
	"4.o_proj": [
	1489,
	1685,
	1708,
	1873,
	2044,
	2079,
	2103,
	2121,
	2629,
	2923,
	3156,
	3272,
	3274,
	3734,
	3834
	],
	"9.o_proj": [
	1676,
	1711,
	1967,
	2112,
	2263,
	2382,
	2510,
	2707,
	2950,
	3438,
	3542,
	3599,
	3741,
	3839,
	3967
	],
	"29.k_proj": [
	1255,
	1455,
	1491,
	1495,
	1700,
	1750,
	2315,
	2376,
	2396,
	2512,
	2513,
	2698,
	3110,
	3732,
	3814
	],
	"7.up_proj": [
	1817,
	1948,
	2077,
	2106,
	2959,
	3049,
	3972,
	4009
	],
	"8.gate_proj": [
	1814,
	2472,
	3094,
	3307,
	3378,
	3537,
	3651,
	3916
	],
	"4.up_proj": [
	1890,
	1915,
	2238,
	2493,
	2519,
	2857,
	2952,
	2989,
	3577,
	3677,
	3757,
	3869
	],
	"30.gate_proj": [
	1887,
	2428,
	2708,
	2823,
	2863,
	2878,
	2920,
	2935,
	3073,
	3390,
	3423,
	3446,
	3615
	],
	"5.v_proj": [
	1693,
	1779,
	2025,
	2150,
	2536,
	2700,
	3218,
	3243,
	3310,
	3344,
	3435,
	3474
	],
	"19.up_proj": [
	1987,
	2315,
	2345,
	2525,
	2581,
	2632,
	2662,
	2692,
	2922,
	2948,
	3199,
	3365,
	3660,
	3729,
	3829,
	3847
	],
	"14.up_proj": [
	1814,
	1961,
	2053,
	2150,
	2457,
	2821,
	2861,
	3014,
	3369,
	3411,
	3594,
	3671,
	3879,
	3932,
	3975
	],
	"16.o_proj": [
	1322,
	1448,
	1588,
	1988,
	2218,
	2401,
	2478,
	2501,
	2730,
	2808,
	3358,
	3558,
	3624
	],
	"29.down_proj": [
	1851,
	1991,
	2065,
	2100,
	2287,
	2409,
	2614,
	2760,
	2889,
	3013,
	3094,
	3122,
	3247,
	3464,
	3636,
	3703,
	3853,
	3854,
	4005,
	4047
	],
	"1.up_proj": [
	1931,
	2243,
	2794,
	3316,
	3460,
	3493,
	4030
	],
	"22.up_proj": [
	1835,
	2072,
	2368,
	2384,
	2688,
	2868,
	2957,
	2967,
	3357,
	3394,
	3530,
	3572,
	3729,
	3932,
	3956
	],
	"14.down_proj": [
	1919,
	2867,
	2902,
	3173,
	3687,
	3794,
	4001,
	4012,
	4043
	],
	"13.v_proj": [
	1440,
	1508,
	1677,
	1814,
	1907,
	2047,
	2378,
	2430,
	2470,
	2613,
	2855,
	2908,
	3012,
	3579,
	3581,
	3617,
	3715,
	4004,
	4082
	],
	"12.gate_proj": [
	2021,
	2353,
	2899,
	2925,
	2964,
	3146,
	3301,
	3391,
	3459,
	3675,
	3963
	],
	"0.down_proj": [
	2195,
	2200,
	2318,
	2458,
	2463,
	2899,
	2921,
	3050,
	3351,
	3369,
	3646,
	4001
	],
	"9.k_proj": [
	1806,
	1980,
	2290,
	2348,
	2440,
	2517,
	2971,
	3294,
	3498
	],
	"18.k_proj": [
	1268,
	1684,
	1889,
	1905,
	1999,
	2087,
	2696,
	2723,
	2758,
	2773,
	2814,
	2871,
	3315,
	3498,
	3525,
	3738,
	3883,
	3970,
	4047,
	4081
	],
	"24.v_proj": [
	1266,
	1622,
	1644,
	1816,
	1995,
	2264,
	2965,
	3161,
	3580,
	3957,
	3968,
	4033
	],
	"30.v_proj": [
	1529,
	1551,
	2107,
	2138,
	2527,
	2580,
	2635,
	2788,
	2996,
	3020,
	3487,
	3496,
	3822,
	3933
	],
	"23.q_proj": [
	1588,
	2160,
	2173,
	2490,
	2494,
	2668,
	2689,
	2965,
	3271,
	3489,
	3607,
	3859,
	4031
	],
	"19.v_proj": [
	1308,
	1647,
	2360,
	2904,
	3027,
	3280,
	3290,
	3385,
	3543,
	3619,
	3774,
	3914,
	3984,
	4047,
	4088
	],
	"13.up_proj": [
	2004,
	2801,
	2804,
	3113,
	3444,
	3506,
	3512,
	3678,
	3923
	],
	"28.down_proj": [
	1835,
	2025,
	2088,
	2314,
	2387,
	2393,
	2514,
	2602,
	2671,
	3075,
	3180,
	3226,
	3355,
	3631,
	3964,
	4031
	],
	"5.k_proj": [
	1404,
	1722,
	1819,
	1983,
	2256,
	2358,
	2675,
	2765,
	2835,
	2949,
	3133,
	3288,
	3308,
	3388,
	3642,
	3766,
	3874,
	3922
	],
	"2.q_proj": [
	1322,
	1334,
	1627,
	2209,
	2244,
	2509,
	2564,
	2961,
	3008,
	3039,
	3110,
	3653,
	3673,
	3736
	],
	"17.up_proj": [
	1830,
	1873,
	2096,
	2115,
	2431,
	2436,
	2593,
	2785,
	2861,
	2993,
	3151,
	3303,
	3323,
	3343,
	3359,
	3362,
	3898,
	3957,
	4081
	],
	"22.o_proj": [
	1400,
	1734,
	1846,
	1877,
	2129,
	2410,
	2486,
	2575,
	2654,
	2760,
	3212,
	3491,
	3683,
	3957,
	3974
	],
	"17.q_proj": [
	1904,
	1963,
	2180,
	2381,
	2610,
	2614,
	2682,
	2696,
	2857,
	2951,
	2982,
	3019,
	3503,
	3554
	],
	"14.k_proj": [
	1615,
	2060,
	2386,
	2565,
	2735,
	2859,
	2883,
	3104,
	3219,
	3256,
	3423,
	3718,
	3818,
	3834
	],
	"25.down_proj": [
	2194,
	2309,
	2759,
	2778,
	2872,
	3063,
	3206,
	3384,
	3404,
	3446,
	3570
	],
	"15.q_proj": [
	1521,
	1568,
	1780,
	1793,
	1931,
	2040,
	2050,
	3114,
	3256,
	3607,
	3625,
	3685,
	3745,
	3955
	],
	"12.k_proj": [
	1359,
	1384,
	1605,
	1626,
	1688,
	2114,
	2356,
	2602,
	2993,
	3023,
	3098,
	3243,
	3914,
	4001,
	4053
	],
	"7.o_proj": [
	1360,
	1517,
	1935,
	2106,
	2110,
	2255,
	2499,
	2957,
	3043,
	3417,
	3594,
	3619,
	3934,
	4058
	],
	"5.up_proj": [
	1930,
	1941,
	1984,
	2092,
	2683,
	2908,
	3061,
	3176,
	3486,
	4039
	],
	"27.q_proj": [
	1489,
	1515,
	1610,
	1622,
	1710,
	1759,
	1811,
	2165,
	2176,
	2292,
	2988,
	3016,
	3092,
	3386,
	3392,
	3579,
	3862,
	3971,
	3977,
	4023,
	4071
	],
	"9.q_proj": [
	1291,
	1307,
	1548,
	1749,
	1968,
	2026,
	2278,
	2403,
	2581,
	3088,
	3289,
	3939
	],
	"0.up_proj": [
	1859,
	1866,
	2252,
	2397,
	2491,
	2500,
	2564,
	2850,
	3073,
	3139,
	3467,
	4080,
	4089
	],
	"4.down_proj": [
	1937,
	2145,
	2368,
	2696,
	2979,
	3143,
	3415,
	3490,
	3590,
	3630,
	3654,
	3722,
	3833,
	3988,
	4059
	],
	"16.k_proj": [
	1402,
	1561,
	1674,
	1914,
	2031,
	3114,
	3518,
	3624,
	3896,
	4017,
	4039
	],
	"30.up_proj": [
	1928,
	2198,
	2271,
	2569,
	2945,
	3176,
	3527,
	4045
	],
	"12.v_proj": [
	1257,
	1368,
	1375,
	1424,
	1481,
	1570,
	1660,
	1773,
	2071,
	2319,
	2989,
	3143,
	3186,
	3375,
	3377,
	3541,
	3567,
	3573,
	3575,
	3584
	],
	"23.gate_proj": [
	2003,
	2158,
	2233,
	2280,
	2398,
	2534,
	2729,
	2820,
	3329,
	3643,
	4063
	],
	"22.q_proj": [
	1299,
	1329,
	1702,
	2371,
	2455,
	2502,
	2545,
	2604,
	2659,
	2662,
	2860,
	3355,
	3443,
	3563,
	3612
	],
	"26.gate_proj": [
	1820,
	2101,
	2799,
	2881,
	3126,
	3407,
	3455,
	3609,
	3762,
	3863
	],
	"30.k_proj": [
	1414,
	1526,
	1864,
	2191,
	2522,
	2627,
	2799,
	2838,
	3084,
	3253,
	3749,
	4007
	],
	"18.gate_proj": [
	1820,
	1853,
	1869,
	1945,
	2123,
	2225,
	2300,
	2434,
	2559,
	2604,
	2765,
	3062,
	3405,
	3810
	],
	"8.up_proj": [
	1793,
	2149,
	2518,
	2656,
	2713,
	2842,
	2893,
	3038,
	3242,
	3315,
	3491,
	3735,
	3741,
	3854,
	3881,
	4089
	],
	"28.up_proj": [
	1851,
	2162,
	2217,
	2278,
	2408,
	2618,
	2658,
	3294,
	3377,
	3607,
	3775,
	3888
	],
	"17.o_proj": [
	1517,
	1680,
	1741,
	1893,
	2259,
	2291,
	2371,
	2792,
	2812,
	3018,
	3364,
	3519,
	3790
	],
	"16.q_proj": [
	1597,
	2151,
	2214,
	2406,
	2584,
	2591,
	3658,
	3715,
	4001,
	4094
	],
	"5.gate_proj": [
	1876,
	2036,
	2057,
	2363,
	2410,
	2525,
	2749,
	2830,
	3095,
	3151,
	3220,
	3410,
	3632,
	3848,
	3870,
	3882
	],
	"23.down_proj": [
	1819,
	2167,
	2265,
	2282,
	2533,
	2706,
	3111,
	3250,
	3449,
	3471,
	3629,
	3657,
	4046
	],
	"13.q_proj": [
	1268,
	1512,
	1885,
	2023,
	2309,
	2343,
	2697,
	2931,
	3072,
	3087,
	3631,
	4084
	],
	"20.down_proj": [
	1842,
	1889,
	2163,
	2359,
	2409,
	2579,
	2741,
	3105,
	3173,
	3195,
	3268,
	3292,
	3832,
	4087
	],
	"6.v_proj": [
	1398,
	1414,
	1751,
	1800,
	1862,
	2105,
	2249,
	2334,
	2509,
	2904,
	3797
	],
	"6.q_proj": [
	1245,
	1300,
	1715,
	1720,
	1774,
	2342,
	2361,
	2833,
	2881,
	2958,
	3440,
	3496,
	3652,
	3899,
	3914
	],
	"3.gate_proj": [
	1813,
	2206,
	2318,
	3126,
	3283,
	3315,
	3479,
	3506,
	3574,
	3597,
	3651,
	3841,
	4004,
	4021,
	4049,
	4059
	],
	"14.v_proj": [
	1292,
	1395,
	1872,
	1895,
	1919,
	1958,
	2078,
	2199,
	2234,
	2480,
	3278,
	4049
	],
	"15.v_proj": [
	1929,
	2069,
	2712,
	2859,
	2905,
	3077,
	3194,
	3380,
	3545,
	3652
	],
	"15.k_proj": [
	1335,
	1465,
	1477,
	1575,
	1717,
	1763,
	2374,
	2802,
	3010,
	3061,
	3256,
	3264,
	3557,
	4000
	],
	"20.k_proj": [
	1466,
	1512,
	1562,
	1922,
	2243,
	2399,
	2546,
	3389,
	3622,
	3850
	],
	"3.q_proj": [
	1419,
	1501,
	1650,
	2329,
	2400,
	2584,
	2923,
	3055,
	3056,
	3243,
	3315,
	3993
	],
	"11.o_proj": [
	1323,
	1520,
	1720,
	2434,
	3014,
	3273,
	3524,
	3561,
	3896,
	3911,
	3992,
	4064
	],
	"2.v_proj": [
	1690,
	1734,
	1939,
	1964,
	2027,
	2070,
	2196,
	2261,
	2424,
	2673,
	2787,
	2829,
	4035
	],
	"22.down_proj": [
	1864,
	2130,
	2186,
	2854,
	3133,
	3266,
	3283,
	3506,
	3536,
	3712,
	3879,
	3956,
	3980,
	4023
	],
	"28.v_proj": [
	1244,
	1260,
	1269,
	1408,
	1454,
	1611,
	1703,
	1865,
	2100,
	2115,
	2130,
	2617,
	2752,
	3380,
	3384,
	3731,
	3883,
	3960,
	4007
	],
	"7.q_proj": [
	1297,
	1319,
	1393,
	1598,
	1723,
	2140,
	2240,
	2475,
	2544,
	2671,
	2977,
	3341,
	3783,
	3829,
	4057,
	4065
	],
	"18.q_proj": [
	1348,
	1656,
	2145,
	2219,
	2591,
	2726,
	2741,
	2798,
	3071,
	3514,
	3526,
	3703,
	3735,
	3737,
	3952,
	3984,
	4059
	],
	"10.o_proj": [
	1368,
	1655,
	1930,
	2161,
	2240,
	2481,
	2499,
	2679,
	2761,
	2803,
	3032,
	3311,
	3338,
	3464,
	3498,
	3671,
	3678,
	3715,
	3738,
	3740,
	4016,
	4077
	],
	"31.gate_proj": [
	1914,
	2056,
	2352,
	2845,
	2989,
	3388,
	3751,
	3807,
	3820,
	4077
	],
	"21.up_proj": [
	1829,
	2081,
	2376,
	2489,
	2700,
	2744,
	2838,
	2949,
	3143,
	3215,
	3387,
	3428,
	3508,
	3607,
	3725,
	4084
	],
	"10.k_proj": [
	1608,
	1950,
	2071,
	2104,
	2117,
	2279,
	2585,
	2660,
	3168,
	3560,
	3805,
	3876,
	3916
	],
	"28.q_proj": [
	1243,
	1544,
	1652,
	1699,
	2251,
	2643,
	2701,
	2751,
	2988,
	3352,
	3508,
	3511,
	3516,
	3567,
	3720
	],
	"4.v_proj": [
	1676,
	1715,
	2263,
	2281,
	2524,
	2584,
	3117,
	3339,
	3964
	],
	"18.down_proj": [
	1902,
	2105,
	2738,
	2778,
	2925,
	2985,
	3139,
	3178,
	3553,
	3571,
	3643,
	3661
	],
	"12.q_proj": [
	1466,
	1492,
	1736,
	1828,
	1946,
	2234,
	2254,
	2356,
	2360,
	2531,
	3184,
	3210,
	3466,
	3889
	],
	"31.o_proj": [
	1383,
	1477,
	1618,
	1737,
	1897,
	2117,
	2225,
	2299,
	2440,
	3023,
	3107,
	3490,
	3689,
	3794,
	4041
	],
	"30.down_proj": [
	1829,
	1838,
	2147,
	2155,
	2671,
	2873,
	2915,
	3398,
	3469,
	3743,
	3765,
	3867,
	3914,
	4033,
	4038,
	4090
	],
	"1.k_proj": [
	1557,
	2014,
	2409,
	2542,
	2816,
	2927,
	3215,
	3846,
	4014
	],
	"27.k_proj": [
	1260,
	1628,
	1686,
	1792,
	2015,
	2787,
	2854,
	2991,
	3372,
	3377,
	3598,
	3679,
	3821
	],
	"25.gate_proj": [
	1991,
	2039,
	2310,
	2413,
	2590,
	3152,
	3454,
	3715
	],
	"9.up_proj": [
	1815,
	1934,
	2053,
	2429,
	2881,
	3097,
	3101,
	3284,
	3330,
	3338,
	3347
	],
	"31.q_proj": [
	1254,
	1305,
	1306,
	1539,
	1762,
	2146,
	2429,
	3065,
	3324,
	3387,
	3625,
	3679,
	3713
	],
	"17.k_proj": [
	1342,
	1395,
	1478,
	1639,
	1766,
	2300,
	2409,
	2437,
	2455,
	2481,
	2554,
	2633,
	3296,
	3311,
	3777
	],
	"23.o_proj": [
	1475,
	1578,
	2102,
	2297,
	2546,
	2764,
	2789,
	3169,
	3339,
	3703,
	3834
	],
	"9.v_proj": [
	1264,
	1425,
	1579,
	1631,
	1667,
	1683,
	2023,
	2632,
	2900,
	3445
	],
	"6.gate_proj": [
	1868,
	1912,
	2343,
	2657,
	2731,
	2826,
	2986,
	3139
	],
	"0.v_proj": [
	1495,
	1728,
	1736,
	1826,
	1852,
	2024,
	2057,
	2089,
	2296,
	2479,
	2530,
	2606,
	2863,
	3064,
	3294,
	3343,
	3514
	],
	"24.q_proj": [
	1523,
	2257,
	2325,
	2394,
	2826,
	2934,
	2975,
	3343,
	3513,
	3712,
	3871
	],
	"8.q_proj": [
	1237,
	1673,
	1726,
	1786,
	1839,
	2033,
	2222,
	2541,
	2771,
	3038,
	3315,
	3513,
	3617,
	3734,
	3770,
	3844,
	3857,
	3982,
	4055
	],
	"31.down_proj": [
	2051,
	2128,
	2295,
	2456,
	2624,
	2890,
	2949,
	2972,
	2979,
	3166,
	3187
	],
	"9.gate_proj": [
	2132,
	2390,
	2784,
	2841,
	3225,
	3304,
	3353,
	3519,
	3658,
	3668,
	3702,
	3845,
	3908
	],
	"21.down_proj": [
	2013,
	2324,
	2662,
	2675,
	3157,
	3356,
	3378,
	3557,
	3583,
	4010
	],
	"16.gate_proj": [
	1964,
	2269,
	2368,
	2504,
	2761,
	2965,
	3149,
	3215,
	3305,
	3570,
	3621,
	3815,
	3928
	],
	"26.down_proj": [
	2057,
	2091,
	2600,
	2705,
	2818,
	2848,
	3166,
	3187,
	3453,
	3525,
	3793,
	3845,
	3987,
	4035,
	4065,
	4067
	],
	"6.down_proj": [
	1915,
	1951,
	1968,
	2239,
	2309,
	2539,
	2653,
	2830,
	2949,
	3003,
	3052,
	3126,
	3508,
	3877,
	3879,
	4038
	],
	"13.down_proj": [
	1942,
	1950,
	2352,
	2405,
	2475,
	3201
	],
	"10.q_proj": [
	1240,
	1567,
	1697,
	2066,
	2308,
	2332,
	2393,
	2473,
	2618,
	2927,
	2995,
	3123,
	3337,
	3346,
	3703
	],
	"30.o_proj": [
	1666,
	1929,
	2177,
	2245,
	2248,
	2320,
	2759,
	2980,
	3030,
	3053,
	3376,
	3378,
	3417,
	4018
	],
	"15.up_proj": [
	1926,
	2089,
	2127,
	2273,
	2302,
	2372,
	2375,
	2824,
	2899,
	2908,
	3145,
	3449,
	3755,
	3766,
	3769,
	3889
	],
	"5.down_proj": [
	1904,
	2249,
	2739,
	2820,
	2917
	],
	"27.down_proj": [
	2280,
	2342,
	2345,
	2588,
	2705,
	2736,
	3024,
	3078,
	3330,
	3382,
	3475,
	3751,
	3827,
	4095
	],
	"15.o_proj": [
	1589,
	1598,
	1683,
	1695,
	1719,
	1746,
	2195,
	2705,
	2748,
	2888,
	3233,
	3444,
	3838,
	3992,
	4091
	],
	"19.q_proj": [
	1314,
	1383,
	1547,
	1658,
	1884,
	1891,
	2056,
	2094,
	2187,
	2188,
	2387,
	2485,
	2539,
	2547,
	2787,
	3109,
	3210,
	3300,
	3442,
	3501,
	3724,
	3856
	],
	"26.v_proj": [
	1564,
	2295,
	2921,
	3452,
	3554,
	3883,
	3983,
	4093
	],
	"0.gate_proj": [
	1796,
	1926,
	2055,
	2286,
	2389,
	2406,
	2628,
	2789,
	3033,
	3081,
	3360,
	3475,
	3610,
	3658,
	3756,
	3788,
	3978
	],
	"1.q_proj": [
	1696,
	1716,
	1930,
	1933,
	1988,
	2140,
	2520,
	2530,
	3079,
	3316,
	3520
	],
	"24.o_proj": [
	1551,
	1788,
	1991,
	1997,
	2088,
	2538,
	2544,
	2601,
	2944,
	3096,
	3368,
	3449,
	3564,
	3571,
	3854
	],
	"25.v_proj": [
	1263,
	1926,
	2439,
	2529,
	2887,
	3000,
	3005,
	3388,
	3418,
	3465,
	3754,
	3790,
	4008,
	4092
	],
	"29.q_proj": [
	1450,
	1821,
	1933,
	1953,
	2102,
	2142,
	2435,
	2483,
	2502,
	2508,
	2596,
	2956,
	3084,
	3911,
	3967,
	3973,
	4061
	],
	"7.v_proj": [
	1665,
	1771,
	1827,
	2065,
	2735,
	3090,
	3526,
	3586,
	3620,
	3908,
	3936
	],
	"8.down_proj": [
	1948,
	2359,
	2571,
	2653,
	2793,
	3042,
	3118,
	3135,
	3415,
	3485,
	3490
	],
	"28.gate_proj": [
	2019,
	2346,
	2414,
	3053,
	3796,
	3804,
	3878,
	4031
	],
	"16.v_proj": [
	1451,
	1655,
	1704,
	1745,
	1970,
	2045,
	2127,
	2613,
	2816,
	3240,
	3387,
	3452,
	3651,
	3677,
	3754,
	3778,
	3830,
	4014
	],
	"27.v_proj": [
	1431,
	1513,
	1781,
	1828,
	1885,
	2109,
	2278,
	2654,
	2791,
	2942,
	3006,
	3053,
	3333,
	3444,
	3457,
	3517,
	3750,
	3774,
	3935
	],
	"19.gate_proj": [
	1943,
	2063,
	2252,
	2430,
	2453,
	2498,
	2544,
	2715,
	2904,
	2965,
	3070,
	3710,
	3867
	],
	"4.q_proj": [
	1906,
	1958,
	1985,
	2063,
	2090,
	2404,
	2798,
	3692,
	3831
	],
	"3.down_proj": [
	1882,
	2129,
	2535,
	2690,
	3453,
	3454,
	3628,
	3649,
	3721
	],
	"20.up_proj": [
	2059,
	2250,
	2431,
	2680,
	2681,
	2826,
	2907,
	2921,
	3070,
	3110,
	3341,
	3458,
	3898,
	3968
	],
	"11.q_proj": [
	1289,
	1594,
	1662,
	1818,
	1985,
	2602,
	2727,
	2883,
	3186,
	3602,
	4000,
	4091
	],
	"11.gate_proj": [
	1959,
	2003,
	2097,
	2283,
	2353,
	2502,
	2730,
	3014,
	3077,
	3161,
	3407,
	3642,
	3660
	],
	"8.k_proj": [
	1473,
	1529,
	1567,
	1713,
	2144,
	2477,
	2530,
	2627,
	3395,
	3434,
	3822,
	4065
	],
	"2.k_proj": [
	1304,
	2289,
	2570,
	2751,
	2770,
	2878,
	3186,
	3196,
	3229,
	3491,
	3496,
	3615
	],
	"14.gate_proj": [
	2045,
	2218,
	2361,
	2471,
	2639,
	3491
	],
	"3.k_proj": [
	1315,
	1362,
	1768,
	1815,
	1816,
	1914,
	2150,
	2494,
	2714,
	2720,
	2835,
	3357
	],
	"21.gate_proj": [
	1869,
	2010,
	2418,
	2555,
	2951,
	3269,
	3430,
	3444,
	3511,
	3663
	],
	"15.gate_proj": [
	2302,
	2391,
	2596,
	2663,
	2842,
	2847,
	2971,
	3273,
	3495,
	3862
	],
	"10.gate_proj": [
	1884,
	1936,
	1954,
	2132,
	2244,
	2472,
	2819,
	2915,
	2922,
	3036,
	3340,
	3406,
	3496,
	3628,
	3857,
	3920
	],
	"28.o_proj": [
	1260,
	1334,
	1355,
	1365,
	1946,
	2340,
	2380,
	2417,
	2611,
	2625,
	2775,
	2995,
	3370,
	3471,
	3489,
	3787,
	3977
	],
	"12.up_proj": [
	1836,
	2014,
	2171,
	2601,
	2646,
	2940,
	3043,
	3235,
	3241,
	3333,
	3361,
	3390,
	3408,
	3706,
	3855,
	3956
	],
	"25.up_proj": [
	1800,
	1980,
	2161,
	2179,
	2254,
	2261,
	2334,
	2407,
	2637,
	2940,
	3354,
	3564,
	3635,
	3791,
	3856
	],
	"17.down_proj": [
	2240,
	2481,
	2627,
	2809,
	2915,
	3053,
	3080,
	3423,
	3613,
	3651,
	3937,
	3950,
	4001,
	4075
	],
	"1.v_proj": [
	1486,
	1805,
	1881,
	2097,
	2311,
	2437,
	2520,
	2544,
	2691,
	3205,
	3233,
	3483,
	3710,
	3827,
	3864
	],
	"18.up_proj": [
	2006,
	2038,
	2147,
	2218,
	2262,
	2265,
	2291,
	2303,
	2338,
	2940,
	3028,
	3087,
	3259,
	3377,
	3642,
	3785,
	3826,
	4052
	],
	"26.q_proj": [
	1645,
	2026,
	2081,
	2575,
	2605,
	2791,
	2840,
	3160,
	3259,
	3448,
	3706,
	3707
	],
	"11.down_proj": [
	1923,
	2028,
	2925,
	2936,
	3093,
	3281,
	3343,
	3373,
	3403,
	3520,
	3576,
	3826,
	3939
	],
	"21.v_proj": [
	1626,
	1944,
	2203,
	2659,
	2795,
	3112,
	3177,
	3392,
	3602,
	4024
	],
	"31.up_proj": [
	1913,
	2321,
	2474,
	2777,
	2837,
	3292,
	3327,
	3623,
	3637,
	3968,
	4043,
	4088
	],
	"20.gate_proj": [
	1830,
	2079,
	2128,
	2547,
	2928,
	3052,
	3206,
	3303,
	3536,
	3624,
	3651,
	3698,
	3914,
	3929,
	3979
	],
	"3.o_proj": [
	1228,
	1255,
	1312,
	1909,
	2531,
	2732,
	2894,
	3136,
	3284,
	3611,
	3664,
	3770,
	4055
	],
	"11.v_proj": [
	1456,
	1849,
	2408,
	2507,
	2626,
	2983,
	3017,
	3143,
	3224,
	3310,
	3445,
	3580,
	3653,
	3867,
	3933,
	4017
	],
	"21.k_proj": [
	1855,
	2214,
	2414,
	2516,
	2754,
	3320,
	3334,
	3362,
	3518,
	3582,
	3711
	],
	"19.down_proj": [
	1828,
	2246,
	2732,
	2926,
	3220,
	3254,
	3286,
	3989
	],
	"31.v_proj": [
	1748,
	1861,
	2061,
	2127,
	2608,
	3243,
	3343,
	3929
	],
	"19.o_proj": [
	1292,
	1807,
	2119,
	2328,
	2514,
	2555,
	2619,
	2780,
	3049,
	3433,
	3589,
	3851
	],
	"5.q_proj": [
	1445,
	1509,
	2689,
	2893,
	2902,
	2973,
	3624,
	3829,
	3890
	],
	"19.k_proj": [
	1255,
	1403,
	1737,
	1876,
	1927,
	2244,
	2284,
	3290,
	3499,
	3817,
	3969
	],
	"2.gate_proj": [
	1969,
	2071,
	2189,
	2193,
	2333,
	2335,
	2372,
	2486,
	2928,
	2947,
	3240,
	3598
	],
	"16.down_proj": [
	1847,
	2688,
	3010,
	3026,
	3135,
	3379,
	3526
	],
	"7.k_proj": [
	1323,
	1560,
	1923,
	2263,
	2472,
	2617,
	2876,
	3290,
	3407,
	3637,
	3717,
	3770
	],
	"6.o_proj": [
	1502,
	1586,
	1656,
	1849,
	2083,
	2097,
	2123,
	2308,
	2341,
	2349,
	2888,
	3023,
	3270,
	3665
	],
	"22.k_proj": [
	1594,
	1968,
	2052,
	2134,
	2643,
	2869,
	3250,
	3256,
	3329,
	3423,
	3518,
	3764,
	3924
	],
	"13.k_proj": [
	1764,
	2144,
	2158,
	2476,
	2783,
	2856,
	2965,
	3031,
	3357,
	3443,
	3594,
	3633,
	3686,
	3687,
	4060
	],
	"10.down_proj": [
	2344,
	2451,
	2659,
	2668,
	3045,
	3175,
	3569,
	3638,
	3672,
	3684,
	3730,
	3971
	],
	"7.gate_proj": [
	2238,
	3592,
	3970,
	4083
	],
	"11.up_proj": [
	1876,
	1963,
	2658,
	2783,
	3254,
	3484,
	3693
	],
	"27.gate_proj": [
	1985,
	2208,
	2520,
	2978,
	3847
	]
	},
	"components_dropped": {}
	}