Hugging Face's logo

defnoteatin
/
pb27-checkpoints

Model card Files
xet
 Community
defnoteatin commited on
Commit
9855ba7
·
verified ·
1 Parent(s): 30e0846

Add flash_next/config.json

Browse files
Files changed (1) hide show
  1. flash_next/config.json +1949 -0
flash_next/config.json ADDED
@@ -0,0 +1,1949 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "PB27Qwen4ExpForConditionalGeneration"
4
+ ],
5
+ "image_token_id": 248056,
6
+ "language_model_only": false,
7
+ "model_type": "qwen4_exp",
8
+ "text_config": {
9
+ "attention_bias": false,
10
+ "attention_dropout": 0.0,
11
+ "bos_token_id": 248044,
12
+ "dtype": "bfloat16",
13
+ "eos_token_id": 248044,
14
+ "full_attention_interval": 4,
15
+ "hc_count": 4,
16
+ "hc_lowrank": 320,
17
+ "head_dim": 256,
18
+ "heads_per_ngram": 8,
19
+ "hidden_act": "silu",
20
+ "hidden_size": 2560,
21
+ "indexer_budget": 2048,
22
+ "indexer_compress_ratio": 4,
23
+ "indexer_head_dim": 128,
24
+ "indexer_kv_heads": 1,
25
+ "indexer_n_heads": 4,
26
+ "initializer_range": 0.02,
27
+ "layer_types": [
28
+ "linear_attention",
29
+ "linear_attention",
30
+ "linear_attention",
31
+ "full_attention",
32
+ "linear_attention",
33
+ "linear_attention",
34
+ "linear_attention",
35
+ "full_attention",
36
+ "linear_attention",
37
+ "linear_attention",
38
+ "linear_attention",
39
+ "full_attention",
40
+ "linear_attention",
41
+ "linear_attention",
42
+ "linear_attention",
43
+ "full_attention",
44
+ "linear_attention",
45
+ "linear_attention",
46
+ "linear_attention",
47
+ "full_attention",
48
+ "linear_attention",
49
+ "linear_attention",
50
+ "linear_attention",
51
+ "full_attention",
52
+ "linear_attention",
53
+ "linear_attention",
54
+ "linear_attention",
55
+ "full_attention",
56
+ "linear_attention",
57
+ "linear_attention",
58
+ "linear_attention",
59
+ "full_attention",
60
+ "linear_attention",
61
+ "linear_attention",
62
+ "linear_attention",
63
+ "full_attention",
64
+ "linear_attention",
65
+ "linear_attention",
66
+ "linear_attention",
67
+ "full_attention",
68
+ "linear_attention",
69
+ "linear_attention",
70
+ "linear_attention",
71
+ "full_attention",
72
+ "linear_attention",
73
+ "linear_attention",
74
+ "linear_attention",
75
+ "full_attention"
76
+ ],
77
+ "linear_conv_kernel_dim": 4,
78
+ "linear_key_head_dim": 128,
79
+ "linear_num_key_heads": 16,
80
+ "linear_num_value_heads": 48,
81
+ "linear_value_head_dim": 128,
82
+ "make_ngram_vocab_size_divisible_by": 128,
83
+ "mamba_ssm_dtype": "float32",
84
+ "max_position_embeddings": 262144,
85
+ "model_type": "qwen4_exp_text",
86
+ "moe_intermediate_size": 640,
87
+ "mtp": {
88
+ "hybrid": true,
89
+ "layer_types": [
90
+ "full_attention"
91
+ ],
92
+ "mtp_use_hidden_state_from_layer": null,
93
+ "num_hidden_layers": 1,
94
+ "rope_theta": 10000000
95
+ },
96
+ "mtp_num_hidden_layers": 1,
97
+ "mtp_use_dedicated_embeddings": false,
98
+ "ngram_size": 3,
99
+ "ngram_vocab_size_base": 20000000,
100
+ "num_attention_heads": 24,
101
+ "num_experts": 512,
102
+ "num_experts_per_tok": 10,
103
+ "num_hidden_layers": 48,
104
+ "num_key_value_heads": 2,
105
+ "output_gate_type": "sigmoid",
106
+ "output_router_logits": false,
107
+ "pad_token_id": null,
108
+ "partial_rotary_factor": 0.25,
109
+ "ple_conv_kernel_size": 4,
110
+ "ple_embed_dim": 2560,
111
+ "ple_layer_ids": [
112
+ 2
113
+ ],
114
+ "rms_norm_eps": 1e-06,
115
+ "rope_parameters": {
116
+ "mrope_interleaved": true,
117
+ "mrope_section": [
118
+ 11,
119
+ 11,
120
+ 10
121
+ ],
122
+ "partial_rotary_factor": 0.25,
123
+ "rope_theta": 10000000,
124
+ "rope_type": "default"
125
+ },
126
+ "router_aux_loss_coef": 0.001,
127
+ "shared_expert_intermediate_size": 640,
128
+ "split_ngram_parts": 128,
129
+ "tie_word_embeddings": false,
130
+ "use_cache": true,
131
+ "vocab_size": 248320,
132
+ "ple_embedding_dtype": "nvfp4",
133
+ "pb_hybrid_layers": [
134
+ 3,
135
+ 7,
136
+ 11,
137
+ 15,
138
+ 19,
139
+ 23,
140
+ 27,
141
+ 31,
142
+ 35,
143
+ 39,
144
+ 43,
145
+ 47
146
+ ],
147
+ "pb_attention_window": 16384,
148
+ "pb_attention_sinks": 4
149
+ },
150
+ "tie_word_embeddings": false,
151
+ "transformers_version": "5.8.0.dev0",
152
+ "video_token_id": 248057,
153
+ "vision_config": {
154
+ "deepstack_visual_indexes": [],
155
+ "depth": 27,
156
+ "hidden_act": "gelu_pytorch_tanh",
157
+ "hidden_size": 1152,
158
+ "in_channels": 3,
159
+ "initializer_range": 0.02,
160
+ "intermediate_size": 4304,
161
+ "model_type": "qwen4_exp",
162
+ "num_heads": 16,
163
+ "num_position_embeddings": 2304,
164
+ "out_hidden_size": 2560,
165
+ "patch_size": 16,
166
+ "spatial_merge_size": 2,
167
+ "temporal_patch_size": 2
168
+ },
169
+ "vision_end_token_id": 248054,
170
+ "vision_start_token_id": 248053,
171
+ "quantization_config": {
172
+ "quant_method": "modelopt",
173
+ "quant_algo": "MIXED_PRECISION",
174
+ "kv_cache_quant_algo": null,
175
+ "ignore": [
176
+ "lm_head",
177
+ "model.visual.*",
178
+ "mtp.fc_embedding",
179
+ "mtp.fc_hidden",
180
+ "mtp.pre_fc_norm_*",
181
+ "mtp.hyper_connection_mixer*",
182
+ "mtp.layers.0.self_attn*",
183
+ "mtp.layers.0.attn_hyper_connection*",
184
+ "mtp.layers.0.mlp_hyper_connection*",
185
+ "mtp.layers.0.mlp.gate",
186
+ "mtp.layers.0.mlp.shared_expert*",
187
+ "*.gdn.*",
188
+ "*.gdn"
189
+ ],
190
+ "quantized_layers": {
191
+ "model.language_model.layers.0.mlp.experts": {
192
+ "quant_algo": "NVFP4",
193
+ "group_size": 16
194
+ },
195
+ "model.language_model.layers.1.mlp.experts": {
196
+ "quant_algo": "NVFP4",
197
+ "group_size": 16
198
+ },
199
+ "model.language_model.layers.2.mlp.experts": {
200
+ "quant_algo": "NVFP4",
201
+ "group_size": 16
202
+ },
203
+ "model.language_model.layers.3.mlp.experts": {
204
+ "quant_algo": "NVFP4",
205
+ "group_size": 16
206
+ },
207
+ "model.language_model.layers.4.mlp.experts": {
208
+ "quant_algo": "NVFP4",
209
+ "group_size": 16
210
+ },
211
+ "model.language_model.layers.5.mlp.experts": {
212
+ "quant_algo": "NVFP4",
213
+ "group_size": 16
214
+ },
215
+ "model.language_model.layers.6.mlp.experts": {
216
+ "quant_algo": "NVFP4",
217
+ "group_size": 16
218
+ },
219
+ "model.language_model.layers.7.mlp.experts": {
220
+ "quant_algo": "NVFP4",
221
+ "group_size": 16
222
+ },
223
+ "model.language_model.layers.8.mlp.experts": {
224
+ "quant_algo": "NVFP4",
225
+ "group_size": 16
226
+ },
227
+ "model.language_model.layers.9.mlp.experts": {
228
+ "quant_algo": "NVFP4",
229
+ "group_size": 16
230
+ },
231
+ "model.language_model.layers.10.mlp.experts": {
232
+ "quant_algo": "NVFP4",
233
+ "group_size": 16
234
+ },
235
+ "model.language_model.layers.11.mlp.experts": {
236
+ "quant_algo": "NVFP4",
237
+ "group_size": 16
238
+ },
239
+ "model.language_model.layers.12.mlp.experts": {
240
+ "quant_algo": "NVFP4",
241
+ "group_size": 16
242
+ },
243
+ "model.language_model.layers.13.mlp.experts": {
244
+ "quant_algo": "NVFP4",
245
+ "group_size": 16
246
+ },
247
+ "model.language_model.layers.14.mlp.experts": {
248
+ "quant_algo": "NVFP4",
249
+ "group_size": 16
250
+ },
251
+ "model.language_model.layers.15.mlp.experts": {
252
+ "quant_algo": "NVFP4",
253
+ "group_size": 16
254
+ },
255
+ "model.language_model.layers.16.mlp.experts": {
256
+ "quant_algo": "NVFP4",
257
+ "group_size": 16
258
+ },
259
+ "model.language_model.layers.17.mlp.experts": {
260
+ "quant_algo": "NVFP4",
261
+ "group_size": 16
262
+ },
263
+ "model.language_model.layers.18.mlp.experts": {
264
+ "quant_algo": "NVFP4",
265
+ "group_size": 16
266
+ },
267
+ "model.language_model.layers.19.mlp.experts": {
268
+ "quant_algo": "NVFP4",
269
+ "group_size": 16
270
+ },
271
+ "model.language_model.layers.20.mlp.experts": {
272
+ "quant_algo": "NVFP4",
273
+ "group_size": 16
274
+ },
275
+ "model.language_model.layers.21.mlp.experts": {
276
+ "quant_algo": "NVFP4",
277
+ "group_size": 16
278
+ },
279
+ "model.language_model.layers.22.mlp.experts": {
280
+ "quant_algo": "NVFP4",
281
+ "group_size": 16
282
+ },
283
+ "model.language_model.layers.23.mlp.experts": {
284
+ "quant_algo": "NVFP4",
285
+ "group_size": 16
286
+ },
287
+ "model.language_model.layers.24.mlp.experts": {
288
+ "quant_algo": "NVFP4",
289
+ "group_size": 16
290
+ },
291
+ "model.language_model.layers.25.mlp.experts": {
292
+ "quant_algo": "NVFP4",
293
+ "group_size": 16
294
+ },
295
+ "model.language_model.layers.26.mlp.experts": {
296
+ "quant_algo": "NVFP4",
297
+ "group_size": 16
298
+ },
299
+ "model.language_model.layers.27.mlp.experts": {
300
+ "quant_algo": "NVFP4",
301
+ "group_size": 16
302
+ },
303
+ "model.language_model.layers.28.mlp.experts": {
304
+ "quant_algo": "NVFP4",
305
+ "group_size": 16
306
+ },
307
+ "model.language_model.layers.29.mlp.experts": {
308
+ "quant_algo": "NVFP4",
309
+ "group_size": 16
310
+ },
311
+ "model.language_model.layers.30.mlp.experts": {
312
+ "quant_algo": "NVFP4",
313
+ "group_size": 16
314
+ },
315
+ "model.language_model.layers.31.mlp.experts": {
316
+ "quant_algo": "NVFP4",
317
+ "group_size": 16
318
+ },
319
+ "model.language_model.layers.32.mlp.experts": {
320
+ "quant_algo": "NVFP4",
321
+ "group_size": 16
322
+ },
323
+ "model.language_model.layers.33.mlp.experts": {
324
+ "quant_algo": "NVFP4",
325
+ "group_size": 16
326
+ },
327
+ "model.language_model.layers.34.mlp.experts": {
328
+ "quant_algo": "NVFP4",
329
+ "group_size": 16
330
+ },
331
+ "model.language_model.layers.35.mlp.experts": {
332
+ "quant_algo": "NVFP4",
333
+ "group_size": 16
334
+ },
335
+ "model.language_model.layers.36.mlp.experts": {
336
+ "quant_algo": "NVFP4",
337
+ "group_size": 16
338
+ },
339
+ "model.language_model.layers.37.mlp.experts": {
340
+ "quant_algo": "NVFP4",
341
+ "group_size": 16
342
+ },
343
+ "model.language_model.layers.38.mlp.experts": {
344
+ "quant_algo": "NVFP4",
345
+ "group_size": 16
346
+ },
347
+ "model.language_model.layers.39.mlp.experts": {
348
+ "quant_algo": "NVFP4",
349
+ "group_size": 16
350
+ },
351
+ "model.language_model.layers.40.mlp.experts": {
352
+ "quant_algo": "NVFP4",
353
+ "group_size": 16
354
+ },
355
+ "model.language_model.layers.41.mlp.experts": {
356
+ "quant_algo": "NVFP4",
357
+ "group_size": 16
358
+ },
359
+ "model.language_model.layers.42.mlp.experts": {
360
+ "quant_algo": "NVFP4",
361
+ "group_size": 16
362
+ },
363
+ "model.language_model.layers.43.mlp.experts": {
364
+ "quant_algo": "NVFP4",
365
+ "group_size": 16
366
+ },
367
+ "model.language_model.layers.44.mlp.experts": {
368
+ "quant_algo": "NVFP4",
369
+ "group_size": 16
370
+ },
371
+ "model.language_model.layers.45.mlp.experts": {
372
+ "quant_algo": "NVFP4",
373
+ "group_size": 16
374
+ },
375
+ "model.language_model.layers.46.mlp.experts": {
376
+ "quant_algo": "NVFP4",
377
+ "group_size": 16
378
+ },
379
+ "model.language_model.layers.47.mlp.experts": {
380
+ "quant_algo": "NVFP4",
381
+ "group_size": 16
382
+ },
383
+ "model.language_model.layers.0.mlp.shared_expert.gate_proj": {
384
+ "quant_algo": "MXFP8",
385
+ "group_size": 32
386
+ },
387
+ "model.language_model.layers.0.mlp.shared_expert.up_proj": {
388
+ "quant_algo": "MXFP8",
389
+ "group_size": 32
390
+ },
391
+ "model.language_model.layers.0.mlp.shared_expert.down_proj": {
392
+ "quant_algo": "MXFP8",
393
+ "group_size": 32
394
+ },
395
+ "model.language_model.layers.1.mlp.shared_expert.gate_proj": {
396
+ "quant_algo": "MXFP8",
397
+ "group_size": 32
398
+ },
399
+ "model.language_model.layers.1.mlp.shared_expert.up_proj": {
400
+ "quant_algo": "MXFP8",
401
+ "group_size": 32
402
+ },
403
+ "model.language_model.layers.1.mlp.shared_expert.down_proj": {
404
+ "quant_algo": "MXFP8",
405
+ "group_size": 32
406
+ },
407
+ "model.language_model.layers.2.mlp.shared_expert.gate_proj": {
408
+ "quant_algo": "MXFP8",
409
+ "group_size": 32
410
+ },
411
+ "model.language_model.layers.2.mlp.shared_expert.up_proj": {
412
+ "quant_algo": "MXFP8",
413
+ "group_size": 32
414
+ },
415
+ "model.language_model.layers.2.mlp.shared_expert.down_proj": {
416
+ "quant_algo": "MXFP8",
417
+ "group_size": 32
418
+ },
419
+ "model.language_model.layers.3.mlp.shared_expert.gate_proj": {
420
+ "quant_algo": "MXFP8",
421
+ "group_size": 32
422
+ },
423
+ "model.language_model.layers.3.mlp.shared_expert.up_proj": {
424
+ "quant_algo": "MXFP8",
425
+ "group_size": 32
426
+ },
427
+ "model.language_model.layers.3.mlp.shared_expert.down_proj": {
428
+ "quant_algo": "MXFP8",
429
+ "group_size": 32
430
+ },
431
+ "model.language_model.layers.4.mlp.shared_expert.gate_proj": {
432
+ "quant_algo": "MXFP8",
433
+ "group_size": 32
434
+ },
435
+ "model.language_model.layers.4.mlp.shared_expert.up_proj": {
436
+ "quant_algo": "MXFP8",
437
+ "group_size": 32
438
+ },
439
+ "model.language_model.layers.4.mlp.shared_expert.down_proj": {
440
+ "quant_algo": "MXFP8",
441
+ "group_size": 32
442
+ },
443
+ "model.language_model.layers.5.mlp.shared_expert.gate_proj": {
444
+ "quant_algo": "MXFP8",
445
+ "group_size": 32
446
+ },
447
+ "model.language_model.layers.5.mlp.shared_expert.up_proj": {
448
+ "quant_algo": "MXFP8",
449
+ "group_size": 32
450
+ },
451
+ "model.language_model.layers.5.mlp.shared_expert.down_proj": {
452
+ "quant_algo": "MXFP8",
453
+ "group_size": 32
454
+ },
455
+ "model.language_model.layers.6.mlp.shared_expert.gate_proj": {
456
+ "quant_algo": "MXFP8",
457
+ "group_size": 32
458
+ },
459
+ "model.language_model.layers.6.mlp.shared_expert.up_proj": {
460
+ "quant_algo": "MXFP8",
461
+ "group_size": 32
462
+ },
463
+ "model.language_model.layers.6.mlp.shared_expert.down_proj": {
464
+ "quant_algo": "MXFP8",
465
+ "group_size": 32
466
+ },
467
+ "model.language_model.layers.7.mlp.shared_expert.gate_proj": {
468
+ "quant_algo": "MXFP8",
469
+ "group_size": 32
470
+ },
471
+ "model.language_model.layers.7.mlp.shared_expert.up_proj": {
472
+ "quant_algo": "MXFP8",
473
+ "group_size": 32
474
+ },
475
+ "model.language_model.layers.7.mlp.shared_expert.down_proj": {
476
+ "quant_algo": "MXFP8",
477
+ "group_size": 32
478
+ },
479
+ "model.language_model.layers.8.mlp.shared_expert.gate_proj": {
480
+ "quant_algo": "MXFP8",
481
+ "group_size": 32
482
+ },
483
+ "model.language_model.layers.8.mlp.shared_expert.up_proj": {
484
+ "quant_algo": "MXFP8",
485
+ "group_size": 32
486
+ },
487
+ "model.language_model.layers.8.mlp.shared_expert.down_proj": {
488
+ "quant_algo": "MXFP8",
489
+ "group_size": 32
490
+ },
491
+ "model.language_model.layers.9.mlp.shared_expert.gate_proj": {
492
+ "quant_algo": "MXFP8",
493
+ "group_size": 32
494
+ },
495
+ "model.language_model.layers.9.mlp.shared_expert.up_proj": {
496
+ "quant_algo": "MXFP8",
497
+ "group_size": 32
498
+ },
499
+ "model.language_model.layers.9.mlp.shared_expert.down_proj": {
500
+ "quant_algo": "MXFP8",
501
+ "group_size": 32
502
+ },
503
+ "model.language_model.layers.10.mlp.shared_expert.gate_proj": {
504
+ "quant_algo": "MXFP8",
505
+ "group_size": 32
506
+ },
507
+ "model.language_model.layers.10.mlp.shared_expert.up_proj": {
508
+ "quant_algo": "MXFP8",
509
+ "group_size": 32
510
+ },
511
+ "model.language_model.layers.10.mlp.shared_expert.down_proj": {
512
+ "quant_algo": "MXFP8",
513
+ "group_size": 32
514
+ },
515
+ "model.language_model.layers.11.mlp.shared_expert.gate_proj": {
516
+ "quant_algo": "MXFP8",
517
+ "group_size": 32
518
+ },
519
+ "model.language_model.layers.11.mlp.shared_expert.up_proj": {
520
+ "quant_algo": "MXFP8",
521
+ "group_size": 32
522
+ },
523
+ "model.language_model.layers.11.mlp.shared_expert.down_proj": {
524
+ "quant_algo": "MXFP8",
525
+ "group_size": 32
526
+ },
527
+ "model.language_model.layers.12.mlp.shared_expert.gate_proj": {
528
+ "quant_algo": "MXFP8",
529
+ "group_size": 32
530
+ },
531
+ "model.language_model.layers.12.mlp.shared_expert.up_proj": {
532
+ "quant_algo": "MXFP8",
533
+ "group_size": 32
534
+ },
535
+ "model.language_model.layers.12.mlp.shared_expert.down_proj": {
536
+ "quant_algo": "MXFP8",
537
+ "group_size": 32
538
+ },
539
+ "model.language_model.layers.13.mlp.shared_expert.gate_proj": {
540
+ "quant_algo": "MXFP8",
541
+ "group_size": 32
542
+ },
543
+ "model.language_model.layers.13.mlp.shared_expert.up_proj": {
544
+ "quant_algo": "MXFP8",
545
+ "group_size": 32
546
+ },
547
+ "model.language_model.layers.13.mlp.shared_expert.down_proj": {
548
+ "quant_algo": "MXFP8",
549
+ "group_size": 32
550
+ },
551
+ "model.language_model.layers.14.mlp.shared_expert.gate_proj": {
552
+ "quant_algo": "MXFP8",
553
+ "group_size": 32
554
+ },
555
+ "model.language_model.layers.14.mlp.shared_expert.up_proj": {
556
+ "quant_algo": "MXFP8",
557
+ "group_size": 32
558
+ },
559
+ "model.language_model.layers.14.mlp.shared_expert.down_proj": {
560
+ "quant_algo": "MXFP8",
561
+ "group_size": 32
562
+ },
563
+ "model.language_model.layers.15.mlp.shared_expert.gate_proj": {
564
+ "quant_algo": "MXFP8",
565
+ "group_size": 32
566
+ },
567
+ "model.language_model.layers.15.mlp.shared_expert.up_proj": {
568
+ "quant_algo": "MXFP8",
569
+ "group_size": 32
570
+ },
571
+ "model.language_model.layers.15.mlp.shared_expert.down_proj": {
572
+ "quant_algo": "MXFP8",
573
+ "group_size": 32
574
+ },
575
+ "model.language_model.layers.16.mlp.shared_expert.gate_proj": {
576
+ "quant_algo": "MXFP8",
577
+ "group_size": 32
578
+ },
579
+ "model.language_model.layers.16.mlp.shared_expert.up_proj": {
580
+ "quant_algo": "MXFP8",
581
+ "group_size": 32
582
+ },
583
+ "model.language_model.layers.16.mlp.shared_expert.down_proj": {
584
+ "quant_algo": "MXFP8",
585
+ "group_size": 32
586
+ },
587
+ "model.language_model.layers.17.mlp.shared_expert.gate_proj": {
588
+ "quant_algo": "MXFP8",
589
+ "group_size": 32
590
+ },
591
+ "model.language_model.layers.17.mlp.shared_expert.up_proj": {
592
+ "quant_algo": "MXFP8",
593
+ "group_size": 32
594
+ },
595
+ "model.language_model.layers.17.mlp.shared_expert.down_proj": {
596
+ "quant_algo": "MXFP8",
597
+ "group_size": 32
598
+ },
599
+ "model.language_model.layers.18.mlp.shared_expert.gate_proj": {
600
+ "quant_algo": "MXFP8",
601
+ "group_size": 32
602
+ },
603
+ "model.language_model.layers.18.mlp.shared_expert.up_proj": {
604
+ "quant_algo": "MXFP8",
605
+ "group_size": 32
606
+ },
607
+ "model.language_model.layers.18.mlp.shared_expert.down_proj": {
608
+ "quant_algo": "MXFP8",
609
+ "group_size": 32
610
+ },
611
+ "model.language_model.layers.19.mlp.shared_expert.gate_proj": {
612
+ "quant_algo": "MXFP8",
613
+ "group_size": 32
614
+ },
615
+ "model.language_model.layers.19.mlp.shared_expert.up_proj": {
616
+ "quant_algo": "MXFP8",
617
+ "group_size": 32
618
+ },
619
+ "model.language_model.layers.19.mlp.shared_expert.down_proj": {
620
+ "quant_algo": "MXFP8",
621
+ "group_size": 32
622
+ },
623
+ "model.language_model.layers.20.mlp.shared_expert.gate_proj": {
624
+ "quant_algo": "MXFP8",
625
+ "group_size": 32
626
+ },
627
+ "model.language_model.layers.20.mlp.shared_expert.up_proj": {
628
+ "quant_algo": "MXFP8",
629
+ "group_size": 32
630
+ },
631
+ "model.language_model.layers.20.mlp.shared_expert.down_proj": {
632
+ "quant_algo": "MXFP8",
633
+ "group_size": 32
634
+ },
635
+ "model.language_model.layers.21.mlp.shared_expert.gate_proj": {
636
+ "quant_algo": "MXFP8",
637
+ "group_size": 32
638
+ },
639
+ "model.language_model.layers.21.mlp.shared_expert.up_proj": {
640
+ "quant_algo": "MXFP8",
641
+ "group_size": 32
642
+ },
643
+ "model.language_model.layers.21.mlp.shared_expert.down_proj": {
644
+ "quant_algo": "MXFP8",
645
+ "group_size": 32
646
+ },
647
+ "model.language_model.layers.22.mlp.shared_expert.gate_proj": {
648
+ "quant_algo": "MXFP8",
649
+ "group_size": 32
650
+ },
651
+ "model.language_model.layers.22.mlp.shared_expert.up_proj": {
652
+ "quant_algo": "MXFP8",
653
+ "group_size": 32
654
+ },
655
+ "model.language_model.layers.22.mlp.shared_expert.down_proj": {
656
+ "quant_algo": "MXFP8",
657
+ "group_size": 32
658
+ },
659
+ "model.language_model.layers.23.mlp.shared_expert.gate_proj": {
660
+ "quant_algo": "MXFP8",
661
+ "group_size": 32
662
+ },
663
+ "model.language_model.layers.23.mlp.shared_expert.up_proj": {
664
+ "quant_algo": "MXFP8",
665
+ "group_size": 32
666
+ },
667
+ "model.language_model.layers.23.mlp.shared_expert.down_proj": {
668
+ "quant_algo": "MXFP8",
669
+ "group_size": 32
670
+ },
671
+ "model.language_model.layers.24.mlp.shared_expert.gate_proj": {
672
+ "quant_algo": "MXFP8",
673
+ "group_size": 32
674
+ },
675
+ "model.language_model.layers.24.mlp.shared_expert.up_proj": {
676
+ "quant_algo": "MXFP8",
677
+ "group_size": 32
678
+ },
679
+ "model.language_model.layers.24.mlp.shared_expert.down_proj": {
680
+ "quant_algo": "MXFP8",
681
+ "group_size": 32
682
+ },
683
+ "model.language_model.layers.25.mlp.shared_expert.gate_proj": {
684
+ "quant_algo": "MXFP8",
685
+ "group_size": 32
686
+ },
687
+ "model.language_model.layers.25.mlp.shared_expert.up_proj": {
688
+ "quant_algo": "MXFP8",
689
+ "group_size": 32
690
+ },
691
+ "model.language_model.layers.25.mlp.shared_expert.down_proj": {
692
+ "quant_algo": "MXFP8",
693
+ "group_size": 32
694
+ },
695
+ "model.language_model.layers.26.mlp.shared_expert.gate_proj": {
696
+ "quant_algo": "MXFP8",
697
+ "group_size": 32
698
+ },
699
+ "model.language_model.layers.26.mlp.shared_expert.up_proj": {
700
+ "quant_algo": "MXFP8",
701
+ "group_size": 32
702
+ },
703
+ "model.language_model.layers.26.mlp.shared_expert.down_proj": {
704
+ "quant_algo": "MXFP8",
705
+ "group_size": 32
706
+ },
707
+ "model.language_model.layers.27.mlp.shared_expert.gate_proj": {
708
+ "quant_algo": "MXFP8",
709
+ "group_size": 32
710
+ },
711
+ "model.language_model.layers.27.mlp.shared_expert.up_proj": {
712
+ "quant_algo": "MXFP8",
713
+ "group_size": 32
714
+ },
715
+ "model.language_model.layers.27.mlp.shared_expert.down_proj": {
716
+ "quant_algo": "MXFP8",
717
+ "group_size": 32
718
+ },
719
+ "model.language_model.layers.28.mlp.shared_expert.gate_proj": {
720
+ "quant_algo": "MXFP8",
721
+ "group_size": 32
722
+ },
723
+ "model.language_model.layers.28.mlp.shared_expert.up_proj": {
724
+ "quant_algo": "MXFP8",
725
+ "group_size": 32
726
+ },
727
+ "model.language_model.layers.28.mlp.shared_expert.down_proj": {
728
+ "quant_algo": "MXFP8",
729
+ "group_size": 32
730
+ },
731
+ "model.language_model.layers.29.mlp.shared_expert.gate_proj": {
732
+ "quant_algo": "MXFP8",
733
+ "group_size": 32
734
+ },
735
+ "model.language_model.layers.29.mlp.shared_expert.up_proj": {
736
+ "quant_algo": "MXFP8",
737
+ "group_size": 32
738
+ },
739
+ "model.language_model.layers.29.mlp.shared_expert.down_proj": {
740
+ "quant_algo": "MXFP8",
741
+ "group_size": 32
742
+ },
743
+ "model.language_model.layers.30.mlp.shared_expert.gate_proj": {
744
+ "quant_algo": "MXFP8",
745
+ "group_size": 32
746
+ },
747
+ "model.language_model.layers.30.mlp.shared_expert.up_proj": {
748
+ "quant_algo": "MXFP8",
749
+ "group_size": 32
750
+ },
751
+ "model.language_model.layers.30.mlp.shared_expert.down_proj": {
752
+ "quant_algo": "MXFP8",
753
+ "group_size": 32
754
+ },
755
+ "model.language_model.layers.31.mlp.shared_expert.gate_proj": {
756
+ "quant_algo": "MXFP8",
757
+ "group_size": 32
758
+ },
759
+ "model.language_model.layers.31.mlp.shared_expert.up_proj": {
760
+ "quant_algo": "MXFP8",
761
+ "group_size": 32
762
+ },
763
+ "model.language_model.layers.31.mlp.shared_expert.down_proj": {
764
+ "quant_algo": "MXFP8",
765
+ "group_size": 32
766
+ },
767
+ "model.language_model.layers.32.mlp.shared_expert.gate_proj": {
768
+ "quant_algo": "MXFP8",
769
+ "group_size": 32
770
+ },
771
+ "model.language_model.layers.32.mlp.shared_expert.up_proj": {
772
+ "quant_algo": "MXFP8",
773
+ "group_size": 32
774
+ },
775
+ "model.language_model.layers.32.mlp.shared_expert.down_proj": {
776
+ "quant_algo": "MXFP8",
777
+ "group_size": 32
778
+ },
779
+ "model.language_model.layers.33.mlp.shared_expert.gate_proj": {
780
+ "quant_algo": "MXFP8",
781
+ "group_size": 32
782
+ },
783
+ "model.language_model.layers.33.mlp.shared_expert.up_proj": {
784
+ "quant_algo": "MXFP8",
785
+ "group_size": 32
786
+ },
787
+ "model.language_model.layers.33.mlp.shared_expert.down_proj": {
788
+ "quant_algo": "MXFP8",
789
+ "group_size": 32
790
+ },
791
+ "model.language_model.layers.34.mlp.shared_expert.gate_proj": {
792
+ "quant_algo": "MXFP8",
793
+ "group_size": 32
794
+ },
795
+ "model.language_model.layers.34.mlp.shared_expert.up_proj": {
796
+ "quant_algo": "MXFP8",
797
+ "group_size": 32
798
+ },
799
+ "model.language_model.layers.34.mlp.shared_expert.down_proj": {
800
+ "quant_algo": "MXFP8",
801
+ "group_size": 32
802
+ },
803
+ "model.language_model.layers.35.mlp.shared_expert.gate_proj": {
804
+ "quant_algo": "MXFP8",
805
+ "group_size": 32
806
+ },
807
+ "model.language_model.layers.35.mlp.shared_expert.up_proj": {
808
+ "quant_algo": "MXFP8",
809
+ "group_size": 32
810
+ },
811
+ "model.language_model.layers.35.mlp.shared_expert.down_proj": {
812
+ "quant_algo": "MXFP8",
813
+ "group_size": 32
814
+ },
815
+ "model.language_model.layers.36.mlp.shared_expert.gate_proj": {
816
+ "quant_algo": "MXFP8",
817
+ "group_size": 32
818
+ },
819
+ "model.language_model.layers.36.mlp.shared_expert.up_proj": {
820
+ "quant_algo": "MXFP8",
821
+ "group_size": 32
822
+ },
823
+ "model.language_model.layers.36.mlp.shared_expert.down_proj": {
824
+ "quant_algo": "MXFP8",
825
+ "group_size": 32
826
+ },
827
+ "model.language_model.layers.37.mlp.shared_expert.gate_proj": {
828
+ "quant_algo": "MXFP8",
829
+ "group_size": 32
830
+ },
831
+ "model.language_model.layers.37.mlp.shared_expert.up_proj": {
832
+ "quant_algo": "MXFP8",
833
+ "group_size": 32
834
+ },
835
+ "model.language_model.layers.37.mlp.shared_expert.down_proj": {
836
+ "quant_algo": "MXFP8",
837
+ "group_size": 32
838
+ },
839
+ "model.language_model.layers.38.mlp.shared_expert.gate_proj": {
840
+ "quant_algo": "MXFP8",
841
+ "group_size": 32
842
+ },
843
+ "model.language_model.layers.38.mlp.shared_expert.up_proj": {
844
+ "quant_algo": "MXFP8",
845
+ "group_size": 32
846
+ },
847
+ "model.language_model.layers.38.mlp.shared_expert.down_proj": {
848
+ "quant_algo": "MXFP8",
849
+ "group_size": 32
850
+ },
851
+ "model.language_model.layers.39.mlp.shared_expert.gate_proj": {
852
+ "quant_algo": "MXFP8",
853
+ "group_size": 32
854
+ },
855
+ "model.language_model.layers.39.mlp.shared_expert.up_proj": {
856
+ "quant_algo": "MXFP8",
857
+ "group_size": 32
858
+ },
859
+ "model.language_model.layers.39.mlp.shared_expert.down_proj": {
860
+ "quant_algo": "MXFP8",
861
+ "group_size": 32
862
+ },
863
+ "model.language_model.layers.40.mlp.shared_expert.gate_proj": {
864
+ "quant_algo": "MXFP8",
865
+ "group_size": 32
866
+ },
867
+ "model.language_model.layers.40.mlp.shared_expert.up_proj": {
868
+ "quant_algo": "MXFP8",
869
+ "group_size": 32
870
+ },
871
+ "model.language_model.layers.40.mlp.shared_expert.down_proj": {
872
+ "quant_algo": "MXFP8",
873
+ "group_size": 32
874
+ },
875
+ "model.language_model.layers.41.mlp.shared_expert.gate_proj": {
876
+ "quant_algo": "MXFP8",
877
+ "group_size": 32
878
+ },
879
+ "model.language_model.layers.41.mlp.shared_expert.up_proj": {
880
+ "quant_algo": "MXFP8",
881
+ "group_size": 32
882
+ },
883
+ "model.language_model.layers.41.mlp.shared_expert.down_proj": {
884
+ "quant_algo": "MXFP8",
885
+ "group_size": 32
886
+ },
887
+ "model.language_model.layers.42.mlp.shared_expert.gate_proj": {
888
+ "quant_algo": "MXFP8",
889
+ "group_size": 32
890
+ },
891
+ "model.language_model.layers.42.mlp.shared_expert.up_proj": {
892
+ "quant_algo": "MXFP8",
893
+ "group_size": 32
894
+ },
895
+ "model.language_model.layers.42.mlp.shared_expert.down_proj": {
896
+ "quant_algo": "MXFP8",
897
+ "group_size": 32
898
+ },
899
+ "model.language_model.layers.43.mlp.shared_expert.gate_proj": {
900
+ "quant_algo": "MXFP8",
901
+ "group_size": 32
902
+ },
903
+ "model.language_model.layers.43.mlp.shared_expert.up_proj": {
904
+ "quant_algo": "MXFP8",
905
+ "group_size": 32
906
+ },
907
+ "model.language_model.layers.43.mlp.shared_expert.down_proj": {
908
+ "quant_algo": "MXFP8",
909
+ "group_size": 32
910
+ },
911
+ "model.language_model.layers.44.mlp.shared_expert.gate_proj": {
912
+ "quant_algo": "MXFP8",
913
+ "group_size": 32
914
+ },
915
+ "model.language_model.layers.44.mlp.shared_expert.up_proj": {
916
+ "quant_algo": "MXFP8",
917
+ "group_size": 32
918
+ },
919
+ "model.language_model.layers.44.mlp.shared_expert.down_proj": {
920
+ "quant_algo": "MXFP8",
921
+ "group_size": 32
922
+ },
923
+ "model.language_model.layers.45.mlp.shared_expert.gate_proj": {
924
+ "quant_algo": "MXFP8",
925
+ "group_size": 32
926
+ },
927
+ "model.language_model.layers.45.mlp.shared_expert.up_proj": {
928
+ "quant_algo": "MXFP8",
929
+ "group_size": 32
930
+ },
931
+ "model.language_model.layers.45.mlp.shared_expert.down_proj": {
932
+ "quant_algo": "MXFP8",
933
+ "group_size": 32
934
+ },
935
+ "model.language_model.layers.46.mlp.shared_expert.gate_proj": {
936
+ "quant_algo": "MXFP8",
937
+ "group_size": 32
938
+ },
939
+ "model.language_model.layers.46.mlp.shared_expert.up_proj": {
940
+ "quant_algo": "MXFP8",
941
+ "group_size": 32
942
+ },
943
+ "model.language_model.layers.46.mlp.shared_expert.down_proj": {
944
+ "quant_algo": "MXFP8",
945
+ "group_size": 32
946
+ },
947
+ "model.language_model.layers.47.mlp.shared_expert.gate_proj": {
948
+ "quant_algo": "MXFP8",
949
+ "group_size": 32
950
+ },
951
+ "model.language_model.layers.47.mlp.shared_expert.up_proj": {
952
+ "quant_algo": "MXFP8",
953
+ "group_size": 32
954
+ },
955
+ "model.language_model.layers.47.mlp.shared_expert.down_proj": {
956
+ "quant_algo": "MXFP8",
957
+ "group_size": 32
958
+ },
959
+ "model.language_model.layers.1.ple.ple_embedding.ngram_embedding": {
960
+ "quant_algo": "NVFP4",
961
+ "group_size": 16
962
+ },
963
+ "mtp.layers.0.mlp.experts": {
964
+ "quant_algo": "W4A16_NVFP4",
965
+ "group_size": 16
966
+ },
967
+ "model.language_model.layers.0.linear_attn.in_proj_a": {
968
+ "quant_algo": "MXFP8",
969
+ "group_size": 32
970
+ },
971
+ "model.language_model.layers.0.linear_attn.in_proj_b": {
972
+ "quant_algo": "MXFP8",
973
+ "group_size": 32
974
+ },
975
+ "model.language_model.layers.0.linear_attn.in_proj_qkv": {
976
+ "quant_algo": "MXFP8",
977
+ "group_size": 32
978
+ },
979
+ "model.language_model.layers.0.linear_attn.in_proj_z": {
980
+ "quant_algo": "MXFP8",
981
+ "group_size": 32
982
+ },
983
+ "model.language_model.layers.0.linear_attn.out_proj": {
984
+ "quant_algo": "MXFP8",
985
+ "group_size": 32
986
+ },
987
+ "model.language_model.layers.1.linear_attn.in_proj_a": {
988
+ "quant_algo": "MXFP8",
989
+ "group_size": 32
990
+ },
991
+ "model.language_model.layers.1.linear_attn.in_proj_b": {
992
+ "quant_algo": "MXFP8",
993
+ "group_size": 32
994
+ },
995
+ "model.language_model.layers.1.linear_attn.in_proj_qkv": {
996
+ "quant_algo": "MXFP8",
997
+ "group_size": 32
998
+ },
999
+ "model.language_model.layers.1.linear_attn.in_proj_z": {
1000
+ "quant_algo": "MXFP8",
1001
+ "group_size": 32
1002
+ },
1003
+ "model.language_model.layers.1.linear_attn.out_proj": {
1004
+ "quant_algo": "MXFP8",
1005
+ "group_size": 32
1006
+ },
1007
+ "model.language_model.layers.10.linear_attn.in_proj_a": {
1008
+ "quant_algo": "MXFP8",
1009
+ "group_size": 32
1010
+ },
1011
+ "model.language_model.layers.10.linear_attn.in_proj_b": {
1012
+ "quant_algo": "MXFP8",
1013
+ "group_size": 32
1014
+ },
1015
+ "model.language_model.layers.10.linear_attn.in_proj_qkv": {
1016
+ "quant_algo": "MXFP8",
1017
+ "group_size": 32
1018
+ },
1019
+ "model.language_model.layers.10.linear_attn.in_proj_z": {
1020
+ "quant_algo": "MXFP8",
1021
+ "group_size": 32
1022
+ },
1023
+ "model.language_model.layers.10.linear_attn.out_proj": {
1024
+ "quant_algo": "MXFP8",
1025
+ "group_size": 32
1026
+ },
1027
+ "model.language_model.layers.11.self_attn.indexer.index_qk_proj": {
1028
+ "quant_algo": "MXFP8",
1029
+ "group_size": 32
1030
+ },
1031
+ "model.language_model.layers.11.self_attn.k_proj": {
1032
+ "quant_algo": "MXFP8",
1033
+ "group_size": 32
1034
+ },
1035
+ "model.language_model.layers.11.self_attn.o_proj": {
1036
+ "quant_algo": "MXFP8",
1037
+ "group_size": 32
1038
+ },
1039
+ "model.language_model.layers.11.self_attn.q_proj": {
1040
+ "quant_algo": "MXFP8",
1041
+ "group_size": 32
1042
+ },
1043
+ "model.language_model.layers.11.self_attn.v_proj": {
1044
+ "quant_algo": "MXFP8",
1045
+ "group_size": 32
1046
+ },
1047
+ "model.language_model.layers.12.linear_attn.in_proj_a": {
1048
+ "quant_algo": "MXFP8",
1049
+ "group_size": 32
1050
+ },
1051
+ "model.language_model.layers.12.linear_attn.in_proj_b": {
1052
+ "quant_algo": "MXFP8",
1053
+ "group_size": 32
1054
+ },
1055
+ "model.language_model.layers.12.linear_attn.in_proj_qkv": {
1056
+ "quant_algo": "MXFP8",
1057
+ "group_size": 32
1058
+ },
1059
+ "model.language_model.layers.12.linear_attn.in_proj_z": {
1060
+ "quant_algo": "MXFP8",
1061
+ "group_size": 32
1062
+ },
1063
+ "model.language_model.layers.12.linear_attn.out_proj": {
1064
+ "quant_algo": "MXFP8",
1065
+ "group_size": 32
1066
+ },
1067
+ "model.language_model.layers.13.linear_attn.in_proj_a": {
1068
+ "quant_algo": "MXFP8",
1069
+ "group_size": 32
1070
+ },
1071
+ "model.language_model.layers.13.linear_attn.in_proj_b": {
1072
+ "quant_algo": "MXFP8",
1073
+ "group_size": 32
1074
+ },
1075
+ "model.language_model.layers.13.linear_attn.in_proj_qkv": {
1076
+ "quant_algo": "MXFP8",
1077
+ "group_size": 32
1078
+ },
1079
+ "model.language_model.layers.13.linear_attn.in_proj_z": {
1080
+ "quant_algo": "MXFP8",
1081
+ "group_size": 32
1082
+ },
1083
+ "model.language_model.layers.13.linear_attn.out_proj": {
1084
+ "quant_algo": "MXFP8",
1085
+ "group_size": 32
1086
+ },
1087
+ "model.language_model.layers.14.linear_attn.in_proj_a": {
1088
+ "quant_algo": "MXFP8",
1089
+ "group_size": 32
1090
+ },
1091
+ "model.language_model.layers.14.linear_attn.in_proj_b": {
1092
+ "quant_algo": "MXFP8",
1093
+ "group_size": 32
1094
+ },
1095
+ "model.language_model.layers.14.linear_attn.in_proj_qkv": {
1096
+ "quant_algo": "MXFP8",
1097
+ "group_size": 32
1098
+ },
1099
+ "model.language_model.layers.14.linear_attn.in_proj_z": {
1100
+ "quant_algo": "MXFP8",
1101
+ "group_size": 32
1102
+ },
1103
+ "model.language_model.layers.14.linear_attn.out_proj": {
1104
+ "quant_algo": "MXFP8",
1105
+ "group_size": 32
1106
+ },
1107
+ "model.language_model.layers.15.self_attn.indexer.index_qk_proj": {
1108
+ "quant_algo": "MXFP8",
1109
+ "group_size": 32
1110
+ },
1111
+ "model.language_model.layers.15.self_attn.k_proj": {
1112
+ "quant_algo": "MXFP8",
1113
+ "group_size": 32
1114
+ },
1115
+ "model.language_model.layers.15.self_attn.o_proj": {
1116
+ "quant_algo": "MXFP8",
1117
+ "group_size": 32
1118
+ },
1119
+ "model.language_model.layers.15.self_attn.q_proj": {
1120
+ "quant_algo": "MXFP8",
1121
+ "group_size": 32
1122
+ },
1123
+ "model.language_model.layers.15.self_attn.v_proj": {
1124
+ "quant_algo": "MXFP8",
1125
+ "group_size": 32
1126
+ },
1127
+ "model.language_model.layers.16.linear_attn.in_proj_a": {
1128
+ "quant_algo": "MXFP8",
1129
+ "group_size": 32
1130
+ },
1131
+ "model.language_model.layers.16.linear_attn.in_proj_b": {
1132
+ "quant_algo": "MXFP8",
1133
+ "group_size": 32
1134
+ },
1135
+ "model.language_model.layers.16.linear_attn.in_proj_qkv": {
1136
+ "quant_algo": "MXFP8",
1137
+ "group_size": 32
1138
+ },
1139
+ "model.language_model.layers.16.linear_attn.in_proj_z": {
1140
+ "quant_algo": "MXFP8",
1141
+ "group_size": 32
1142
+ },
1143
+ "model.language_model.layers.16.linear_attn.out_proj": {
1144
+ "quant_algo": "MXFP8",
1145
+ "group_size": 32
1146
+ },
1147
+ "model.language_model.layers.17.linear_attn.in_proj_a": {
1148
+ "quant_algo": "MXFP8",
1149
+ "group_size": 32
1150
+ },
1151
+ "model.language_model.layers.17.linear_attn.in_proj_b": {
1152
+ "quant_algo": "MXFP8",
1153
+ "group_size": 32
1154
+ },
1155
+ "model.language_model.layers.17.linear_attn.in_proj_qkv": {
1156
+ "quant_algo": "MXFP8",
1157
+ "group_size": 32
1158
+ },
1159
+ "model.language_model.layers.17.linear_attn.in_proj_z": {
1160
+ "quant_algo": "MXFP8",
1161
+ "group_size": 32
1162
+ },
1163
+ "model.language_model.layers.17.linear_attn.out_proj": {
1164
+ "quant_algo": "MXFP8",
1165
+ "group_size": 32
1166
+ },
1167
+ "model.language_model.layers.18.linear_attn.in_proj_a": {
1168
+ "quant_algo": "MXFP8",
1169
+ "group_size": 32
1170
+ },
1171
+ "model.language_model.layers.18.linear_attn.in_proj_b": {
1172
+ "quant_algo": "MXFP8",
1173
+ "group_size": 32
1174
+ },
1175
+ "model.language_model.layers.18.linear_attn.in_proj_qkv": {
1176
+ "quant_algo": "MXFP8",
1177
+ "group_size": 32
1178
+ },
1179
+ "model.language_model.layers.18.linear_attn.in_proj_z": {
1180
+ "quant_algo": "MXFP8",
1181
+ "group_size": 32
1182
+ },
1183
+ "model.language_model.layers.18.linear_attn.out_proj": {
1184
+ "quant_algo": "MXFP8",
1185
+ "group_size": 32
1186
+ },
1187
+ "model.language_model.layers.19.self_attn.indexer.index_qk_proj": {
1188
+ "quant_algo": "MXFP8",
1189
+ "group_size": 32
1190
+ },
1191
+ "model.language_model.layers.19.self_attn.k_proj": {
1192
+ "quant_algo": "MXFP8",
1193
+ "group_size": 32
1194
+ },
1195
+ "model.language_model.layers.19.self_attn.o_proj": {
1196
+ "quant_algo": "MXFP8",
1197
+ "group_size": 32
1198
+ },
1199
+ "model.language_model.layers.19.self_attn.q_proj": {
1200
+ "quant_algo": "MXFP8",
1201
+ "group_size": 32
1202
+ },
1203
+ "model.language_model.layers.19.self_attn.v_proj": {
1204
+ "quant_algo": "MXFP8",
1205
+ "group_size": 32
1206
+ },
1207
+ "model.language_model.layers.2.linear_attn.in_proj_a": {
1208
+ "quant_algo": "MXFP8",
1209
+ "group_size": 32
1210
+ },
1211
+ "model.language_model.layers.2.linear_attn.in_proj_b": {
1212
+ "quant_algo": "MXFP8",
1213
+ "group_size": 32
1214
+ },
1215
+ "model.language_model.layers.2.linear_attn.in_proj_qkv": {
1216
+ "quant_algo": "MXFP8",
1217
+ "group_size": 32
1218
+ },
1219
+ "model.language_model.layers.2.linear_attn.in_proj_z": {
1220
+ "quant_algo": "MXFP8",
1221
+ "group_size": 32
1222
+ },
1223
+ "model.language_model.layers.2.linear_attn.out_proj": {
1224
+ "quant_algo": "MXFP8",
1225
+ "group_size": 32
1226
+ },
1227
+ "model.language_model.layers.20.linear_attn.in_proj_a": {
1228
+ "quant_algo": "MXFP8",
1229
+ "group_size": 32
1230
+ },
1231
+ "model.language_model.layers.20.linear_attn.in_proj_b": {
1232
+ "quant_algo": "MXFP8",
1233
+ "group_size": 32
1234
+ },
1235
+ "model.language_model.layers.20.linear_attn.in_proj_qkv": {
1236
+ "quant_algo": "MXFP8",
1237
+ "group_size": 32
1238
+ },
1239
+ "model.language_model.layers.20.linear_attn.in_proj_z": {
1240
+ "quant_algo": "MXFP8",
1241
+ "group_size": 32
1242
+ },
1243
+ "model.language_model.layers.20.linear_attn.out_proj": {
1244
+ "quant_algo": "MXFP8",
1245
+ "group_size": 32
1246
+ },
1247
+ "model.language_model.layers.21.linear_attn.in_proj_a": {
1248
+ "quant_algo": "MXFP8",
1249
+ "group_size": 32
1250
+ },
1251
+ "model.language_model.layers.21.linear_attn.in_proj_b": {
1252
+ "quant_algo": "MXFP8",
1253
+ "group_size": 32
1254
+ },
1255
+ "model.language_model.layers.21.linear_attn.in_proj_qkv": {
1256
+ "quant_algo": "MXFP8",
1257
+ "group_size": 32
1258
+ },
1259
+ "model.language_model.layers.21.linear_attn.in_proj_z": {
1260
+ "quant_algo": "MXFP8",
1261
+ "group_size": 32
1262
+ },
1263
+ "model.language_model.layers.21.linear_attn.out_proj": {
1264
+ "quant_algo": "MXFP8",
1265
+ "group_size": 32
1266
+ },
1267
+ "model.language_model.layers.22.linear_attn.in_proj_a": {
1268
+ "quant_algo": "MXFP8",
1269
+ "group_size": 32
1270
+ },
1271
+ "model.language_model.layers.22.linear_attn.in_proj_b": {
1272
+ "quant_algo": "MXFP8",
1273
+ "group_size": 32
1274
+ },
1275
+ "model.language_model.layers.22.linear_attn.in_proj_qkv": {
1276
+ "quant_algo": "MXFP8",
1277
+ "group_size": 32
1278
+ },
1279
+ "model.language_model.layers.22.linear_attn.in_proj_z": {
1280
+ "quant_algo": "MXFP8",
1281
+ "group_size": 32
1282
+ },
1283
+ "model.language_model.layers.22.linear_attn.out_proj": {
1284
+ "quant_algo": "MXFP8",
1285
+ "group_size": 32
1286
+ },
1287
+ "model.language_model.layers.23.self_attn.indexer.index_qk_proj": {
1288
+ "quant_algo": "MXFP8",
1289
+ "group_size": 32
1290
+ },
1291
+ "model.language_model.layers.23.self_attn.k_proj": {
1292
+ "quant_algo": "MXFP8",
1293
+ "group_size": 32
1294
+ },
1295
+ "model.language_model.layers.23.self_attn.o_proj": {
1296
+ "quant_algo": "MXFP8",
1297
+ "group_size": 32
1298
+ },
1299
+ "model.language_model.layers.23.self_attn.q_proj": {
1300
+ "quant_algo": "MXFP8",
1301
+ "group_size": 32
1302
+ },
1303
+ "model.language_model.layers.23.self_attn.v_proj": {
1304
+ "quant_algo": "MXFP8",
1305
+ "group_size": 32
1306
+ },
1307
+ "model.language_model.layers.24.linear_attn.in_proj_a": {
1308
+ "quant_algo": "MXFP8",
1309
+ "group_size": 32
1310
+ },
1311
+ "model.language_model.layers.24.linear_attn.in_proj_b": {
1312
+ "quant_algo": "MXFP8",
1313
+ "group_size": 32
1314
+ },
1315
+ "model.language_model.layers.24.linear_attn.in_proj_qkv": {
1316
+ "quant_algo": "MXFP8",
1317
+ "group_size": 32
1318
+ },
1319
+ "model.language_model.layers.24.linear_attn.in_proj_z": {
1320
+ "quant_algo": "MXFP8",
1321
+ "group_size": 32
1322
+ },
1323
+ "model.language_model.layers.24.linear_attn.out_proj": {
1324
+ "quant_algo": "MXFP8",
1325
+ "group_size": 32
1326
+ },
1327
+ "model.language_model.layers.25.linear_attn.in_proj_a": {
1328
+ "quant_algo": "MXFP8",
1329
+ "group_size": 32
1330
+ },
1331
+ "model.language_model.layers.25.linear_attn.in_proj_b": {
1332
+ "quant_algo": "MXFP8",
1333
+ "group_size": 32
1334
+ },
1335
+ "model.language_model.layers.25.linear_attn.in_proj_qkv": {
1336
+ "quant_algo": "MXFP8",
1337
+ "group_size": 32
1338
+ },
1339
+ "model.language_model.layers.25.linear_attn.in_proj_z": {
1340
+ "quant_algo": "MXFP8",
1341
+ "group_size": 32
1342
+ },
1343
+ "model.language_model.layers.25.linear_attn.out_proj": {
1344
+ "quant_algo": "MXFP8",
1345
+ "group_size": 32
1346
+ },
1347
+ "model.language_model.layers.26.linear_attn.in_proj_a": {
1348
+ "quant_algo": "MXFP8",
1349
+ "group_size": 32
1350
+ },
1351
+ "model.language_model.layers.26.linear_attn.in_proj_b": {
1352
+ "quant_algo": "MXFP8",
1353
+ "group_size": 32
1354
+ },
1355
+ "model.language_model.layers.26.linear_attn.in_proj_qkv": {
1356
+ "quant_algo": "MXFP8",
1357
+ "group_size": 32
1358
+ },
1359
+ "model.language_model.layers.26.linear_attn.in_proj_z": {
1360
+ "quant_algo": "MXFP8",
1361
+ "group_size": 32
1362
+ },
1363
+ "model.language_model.layers.26.linear_attn.out_proj": {
1364
+ "quant_algo": "MXFP8",
1365
+ "group_size": 32
1366
+ },
1367
+ "model.language_model.layers.27.self_attn.indexer.index_qk_proj": {
1368
+ "quant_algo": "MXFP8",
1369
+ "group_size": 32
1370
+ },
1371
+ "model.language_model.layers.27.self_attn.k_proj": {
1372
+ "quant_algo": "MXFP8",
1373
+ "group_size": 32
1374
+ },
1375
+ "model.language_model.layers.27.self_attn.o_proj": {
1376
+ "quant_algo": "MXFP8",
1377
+ "group_size": 32
1378
+ },
1379
+ "model.language_model.layers.27.self_attn.q_proj": {
1380
+ "quant_algo": "MXFP8",
1381
+ "group_size": 32
1382
+ },
1383
+ "model.language_model.layers.27.self_attn.v_proj": {
1384
+ "quant_algo": "MXFP8",
1385
+ "group_size": 32
1386
+ },
1387
+ "model.language_model.layers.28.linear_attn.in_proj_a": {
1388
+ "quant_algo": "MXFP8",
1389
+ "group_size": 32
1390
+ },
1391
+ "model.language_model.layers.28.linear_attn.in_proj_b": {
1392
+ "quant_algo": "MXFP8",
1393
+ "group_size": 32
1394
+ },
1395
+ "model.language_model.layers.28.linear_attn.in_proj_qkv": {
1396
+ "quant_algo": "MXFP8",
1397
+ "group_size": 32
1398
+ },
1399
+ "model.language_model.layers.28.linear_attn.in_proj_z": {
1400
+ "quant_algo": "MXFP8",
1401
+ "group_size": 32
1402
+ },
1403
+ "model.language_model.layers.28.linear_attn.out_proj": {
1404
+ "quant_algo": "MXFP8",
1405
+ "group_size": 32
1406
+ },
1407
+ "model.language_model.layers.29.linear_attn.in_proj_a": {
1408
+ "quant_algo": "MXFP8",
1409
+ "group_size": 32
1410
+ },
1411
+ "model.language_model.layers.29.linear_attn.in_proj_b": {
1412
+ "quant_algo": "MXFP8",
1413
+ "group_size": 32
1414
+ },
1415
+ "model.language_model.layers.29.linear_attn.in_proj_qkv": {
1416
+ "quant_algo": "MXFP8",
1417
+ "group_size": 32
1418
+ },
1419
+ "model.language_model.layers.29.linear_attn.in_proj_z": {
1420
+ "quant_algo": "MXFP8",
1421
+ "group_size": 32
1422
+ },
1423
+ "model.language_model.layers.29.linear_attn.out_proj": {
1424
+ "quant_algo": "MXFP8",
1425
+ "group_size": 32
1426
+ },
1427
+ "model.language_model.layers.3.self_attn.indexer.index_qk_proj": {
1428
+ "quant_algo": "MXFP8",
1429
+ "group_size": 32
1430
+ },
1431
+ "model.language_model.layers.3.self_attn.k_proj": {
1432
+ "quant_algo": "MXFP8",
1433
+ "group_size": 32
1434
+ },
1435
+ "model.language_model.layers.3.self_attn.o_proj": {
1436
+ "quant_algo": "MXFP8",
1437
+ "group_size": 32
1438
+ },
1439
+ "model.language_model.layers.3.self_attn.q_proj": {
1440
+ "quant_algo": "MXFP8",
1441
+ "group_size": 32
1442
+ },
1443
+ "model.language_model.layers.3.self_attn.v_proj": {
1444
+ "quant_algo": "MXFP8",
1445
+ "group_size": 32
1446
+ },
1447
+ "model.language_model.layers.30.linear_attn.in_proj_a": {
1448
+ "quant_algo": "MXFP8",
1449
+ "group_size": 32
1450
+ },
1451
+ "model.language_model.layers.30.linear_attn.in_proj_b": {
1452
+ "quant_algo": "MXFP8",
1453
+ "group_size": 32
1454
+ },
1455
+ "model.language_model.layers.30.linear_attn.in_proj_qkv": {
1456
+ "quant_algo": "MXFP8",
1457
+ "group_size": 32
1458
+ },
1459
+ "model.language_model.layers.30.linear_attn.in_proj_z": {
1460
+ "quant_algo": "MXFP8",
1461
+ "group_size": 32
1462
+ },
1463
+ "model.language_model.layers.30.linear_attn.out_proj": {
1464
+ "quant_algo": "MXFP8",
1465
+ "group_size": 32
1466
+ },
1467
+ "model.language_model.layers.31.self_attn.indexer.index_qk_proj": {
1468
+ "quant_algo": "MXFP8",
1469
+ "group_size": 32
1470
+ },
1471
+ "model.language_model.layers.31.self_attn.k_proj": {
1472
+ "quant_algo": "MXFP8",
1473
+ "group_size": 32
1474
+ },
1475
+ "model.language_model.layers.31.self_attn.o_proj": {
1476
+ "quant_algo": "MXFP8",
1477
+ "group_size": 32
1478
+ },
1479
+ "model.language_model.layers.31.self_attn.q_proj": {
1480
+ "quant_algo": "MXFP8",
1481
+ "group_size": 32
1482
+ },
1483
+ "model.language_model.layers.31.self_attn.v_proj": {
1484
+ "quant_algo": "MXFP8",
1485
+ "group_size": 32
1486
+ },
1487
+ "model.language_model.layers.32.linear_attn.in_proj_a": {
1488
+ "quant_algo": "MXFP8",
1489
+ "group_size": 32
1490
+ },
1491
+ "model.language_model.layers.32.linear_attn.in_proj_b": {
1492
+ "quant_algo": "MXFP8",
1493
+ "group_size": 32
1494
+ },
1495
+ "model.language_model.layers.32.linear_attn.in_proj_qkv": {
1496
+ "quant_algo": "MXFP8",
1497
+ "group_size": 32
1498
+ },
1499
+ "model.language_model.layers.32.linear_attn.in_proj_z": {
1500
+ "quant_algo": "MXFP8",
1501
+ "group_size": 32
1502
+ },
1503
+ "model.language_model.layers.32.linear_attn.out_proj": {
1504
+ "quant_algo": "MXFP8",
1505
+ "group_size": 32
1506
+ },
1507
+ "model.language_model.layers.33.linear_attn.in_proj_a": {
1508
+ "quant_algo": "MXFP8",
1509
+ "group_size": 32
1510
+ },
1511
+ "model.language_model.layers.33.linear_attn.in_proj_b": {
1512
+ "quant_algo": "MXFP8",
1513
+ "group_size": 32
1514
+ },
1515
+ "model.language_model.layers.33.linear_attn.in_proj_qkv": {
1516
+ "quant_algo": "MXFP8",
1517
+ "group_size": 32
1518
+ },
1519
+ "model.language_model.layers.33.linear_attn.in_proj_z": {
1520
+ "quant_algo": "MXFP8",
1521
+ "group_size": 32
1522
+ },
1523
+ "model.language_model.layers.33.linear_attn.out_proj": {
1524
+ "quant_algo": "MXFP8",
1525
+ "group_size": 32
1526
+ },
1527
+ "model.language_model.layers.34.linear_attn.in_proj_a": {
1528
+ "quant_algo": "MXFP8",
1529
+ "group_size": 32
1530
+ },
1531
+ "model.language_model.layers.34.linear_attn.in_proj_b": {
1532
+ "quant_algo": "MXFP8",
1533
+ "group_size": 32
1534
+ },
1535
+ "model.language_model.layers.34.linear_attn.in_proj_qkv": {
1536
+ "quant_algo": "MXFP8",
1537
+ "group_size": 32
1538
+ },
1539
+ "model.language_model.layers.34.linear_attn.in_proj_z": {
1540
+ "quant_algo": "MXFP8",
1541
+ "group_size": 32
1542
+ },
1543
+ "model.language_model.layers.34.linear_attn.out_proj": {
1544
+ "quant_algo": "MXFP8",
1545
+ "group_size": 32
1546
+ },
1547
+ "model.language_model.layers.35.self_attn.indexer.index_qk_proj": {
1548
+ "quant_algo": "MXFP8",
1549
+ "group_size": 32
1550
+ },
1551
+ "model.language_model.layers.35.self_attn.k_proj": {
1552
+ "quant_algo": "MXFP8",
1553
+ "group_size": 32
1554
+ },
1555
+ "model.language_model.layers.35.self_attn.o_proj": {
1556
+ "quant_algo": "MXFP8",
1557
+ "group_size": 32
1558
+ },
1559
+ "model.language_model.layers.35.self_attn.q_proj": {
1560
+ "quant_algo": "MXFP8",
1561
+ "group_size": 32
1562
+ },
1563
+ "model.language_model.layers.35.self_attn.v_proj": {
1564
+ "quant_algo": "MXFP8",
1565
+ "group_size": 32
1566
+ },
1567
+ "model.language_model.layers.36.linear_attn.in_proj_a": {
1568
+ "quant_algo": "MXFP8",
1569
+ "group_size": 32
1570
+ },
1571
+ "model.language_model.layers.36.linear_attn.in_proj_b": {
1572
+ "quant_algo": "MXFP8",
1573
+ "group_size": 32
1574
+ },
1575
+ "model.language_model.layers.36.linear_attn.in_proj_qkv": {
1576
+ "quant_algo": "MXFP8",
1577
+ "group_size": 32
1578
+ },
1579
+ "model.language_model.layers.36.linear_attn.in_proj_z": {
1580
+ "quant_algo": "MXFP8",
1581
+ "group_size": 32
1582
+ },
1583
+ "model.language_model.layers.36.linear_attn.out_proj": {
1584
+ "quant_algo": "MXFP8",
1585
+ "group_size": 32
1586
+ },
1587
+ "model.language_model.layers.37.linear_attn.in_proj_a": {
1588
+ "quant_algo": "MXFP8",
1589
+ "group_size": 32
1590
+ },
1591
+ "model.language_model.layers.37.linear_attn.in_proj_b": {
1592
+ "quant_algo": "MXFP8",
1593
+ "group_size": 32
1594
+ },
1595
+ "model.language_model.layers.37.linear_attn.in_proj_qkv": {
1596
+ "quant_algo": "MXFP8",
1597
+ "group_size": 32
1598
+ },
1599
+ "model.language_model.layers.37.linear_attn.in_proj_z": {
1600
+ "quant_algo": "MXFP8",
1601
+ "group_size": 32
1602
+ },
1603
+ "model.language_model.layers.37.linear_attn.out_proj": {
1604
+ "quant_algo": "MXFP8",
1605
+ "group_size": 32
1606
+ },
1607
+ "model.language_model.layers.38.linear_attn.in_proj_a": {
1608
+ "quant_algo": "MXFP8",
1609
+ "group_size": 32
1610
+ },
1611
+ "model.language_model.layers.38.linear_attn.in_proj_b": {
1612
+ "quant_algo": "MXFP8",
1613
+ "group_size": 32
1614
+ },
1615
+ "model.language_model.layers.38.linear_attn.in_proj_qkv": {
1616
+ "quant_algo": "MXFP8",
1617
+ "group_size": 32
1618
+ },
1619
+ "model.language_model.layers.38.linear_attn.in_proj_z": {
1620
+ "quant_algo": "MXFP8",
1621
+ "group_size": 32
1622
+ },
1623
+ "model.language_model.layers.38.linear_attn.out_proj": {
1624
+ "quant_algo": "MXFP8",
1625
+ "group_size": 32
1626
+ },
1627
+ "model.language_model.layers.39.self_attn.indexer.index_qk_proj": {
1628
+ "quant_algo": "MXFP8",
1629
+ "group_size": 32
1630
+ },
1631
+ "model.language_model.layers.39.self_attn.k_proj": {
1632
+ "quant_algo": "MXFP8",
1633
+ "group_size": 32
1634
+ },
1635
+ "model.language_model.layers.39.self_attn.o_proj": {
1636
+ "quant_algo": "MXFP8",
1637
+ "group_size": 32
1638
+ },
1639
+ "model.language_model.layers.39.self_attn.q_proj": {
1640
+ "quant_algo": "MXFP8",
1641
+ "group_size": 32
1642
+ },
1643
+ "model.language_model.layers.39.self_attn.v_proj": {
1644
+ "quant_algo": "MXFP8",
1645
+ "group_size": 32
1646
+ },
1647
+ "model.language_model.layers.4.linear_attn.in_proj_a": {
1648
+ "quant_algo": "MXFP8",
1649
+ "group_size": 32
1650
+ },
1651
+ "model.language_model.layers.4.linear_attn.in_proj_b": {
1652
+ "quant_algo": "MXFP8",
1653
+ "group_size": 32
1654
+ },
1655
+ "model.language_model.layers.4.linear_attn.in_proj_qkv": {
1656
+ "quant_algo": "MXFP8",
1657
+ "group_size": 32
1658
+ },
1659
+ "model.language_model.layers.4.linear_attn.in_proj_z": {
1660
+ "quant_algo": "MXFP8",
1661
+ "group_size": 32
1662
+ },
1663
+ "model.language_model.layers.4.linear_attn.out_proj": {
1664
+ "quant_algo": "MXFP8",
1665
+ "group_size": 32
1666
+ },
1667
+ "model.language_model.layers.40.linear_attn.in_proj_a": {
1668
+ "quant_algo": "MXFP8",
1669
+ "group_size": 32
1670
+ },
1671
+ "model.language_model.layers.40.linear_attn.in_proj_b": {
1672
+ "quant_algo": "MXFP8",
1673
+ "group_size": 32
1674
+ },
1675
+ "model.language_model.layers.40.linear_attn.in_proj_qkv": {
1676
+ "quant_algo": "MXFP8",
1677
+ "group_size": 32
1678
+ },
1679
+ "model.language_model.layers.40.linear_attn.in_proj_z": {
1680
+ "quant_algo": "MXFP8",
1681
+ "group_size": 32
1682
+ },
1683
+ "model.language_model.layers.40.linear_attn.out_proj": {
1684
+ "quant_algo": "MXFP8",
1685
+ "group_size": 32
1686
+ },
1687
+ "model.language_model.layers.41.linear_attn.in_proj_a": {
1688
+ "quant_algo": "MXFP8",
1689
+ "group_size": 32
1690
+ },
1691
+ "model.language_model.layers.41.linear_attn.in_proj_b": {
1692
+ "quant_algo": "MXFP8",
1693
+ "group_size": 32
1694
+ },
1695
+ "model.language_model.layers.41.linear_attn.in_proj_qkv": {
1696
+ "quant_algo": "MXFP8",
1697
+ "group_size": 32
1698
+ },
1699
+ "model.language_model.layers.41.linear_attn.in_proj_z": {
1700
+ "quant_algo": "MXFP8",
1701
+ "group_size": 32
1702
+ },
1703
+ "model.language_model.layers.41.linear_attn.out_proj": {
1704
+ "quant_algo": "MXFP8",
1705
+ "group_size": 32
1706
+ },
1707
+ "model.language_model.layers.42.linear_attn.in_proj_a": {
1708
+ "quant_algo": "MXFP8",
1709
+ "group_size": 32
1710
+ },
1711
+ "model.language_model.layers.42.linear_attn.in_proj_b": {
1712
+ "quant_algo": "MXFP8",
1713
+ "group_size": 32
1714
+ },
1715
+ "model.language_model.layers.42.linear_attn.in_proj_qkv": {
1716
+ "quant_algo": "MXFP8",
1717
+ "group_size": 32
1718
+ },
1719
+ "model.language_model.layers.42.linear_attn.in_proj_z": {
1720
+ "quant_algo": "MXFP8",
1721
+ "group_size": 32
1722
+ },
1723
+ "model.language_model.layers.42.linear_attn.out_proj": {
1724
+ "quant_algo": "MXFP8",
1725
+ "group_size": 32
1726
+ },
1727
+ "model.language_model.layers.43.self_attn.indexer.index_qk_proj": {
1728
+ "quant_algo": "MXFP8",
1729
+ "group_size": 32
1730
+ },
1731
+ "model.language_model.layers.43.self_attn.k_proj": {
1732
+ "quant_algo": "MXFP8",
1733
+ "group_size": 32
1734
+ },
1735
+ "model.language_model.layers.43.self_attn.o_proj": {
1736
+ "quant_algo": "MXFP8",
1737
+ "group_size": 32
1738
+ },
1739
+ "model.language_model.layers.43.self_attn.q_proj": {
1740
+ "quant_algo": "MXFP8",
1741
+ "group_size": 32
1742
+ },
1743
+ "model.language_model.layers.43.self_attn.v_proj": {
1744
+ "quant_algo": "MXFP8",
1745
+ "group_size": 32
1746
+ },
1747
+ "model.language_model.layers.44.linear_attn.in_proj_a": {
1748
+ "quant_algo": "MXFP8",
1749
+ "group_size": 32
1750
+ },
1751
+ "model.language_model.layers.44.linear_attn.in_proj_b": {
1752
+ "quant_algo": "MXFP8",
1753
+ "group_size": 32
1754
+ },
1755
+ "model.language_model.layers.44.linear_attn.in_proj_qkv": {
1756
+ "quant_algo": "MXFP8",
1757
+ "group_size": 32
1758
+ },
1759
+ "model.language_model.layers.44.linear_attn.in_proj_z": {
1760
+ "quant_algo": "MXFP8",
1761
+ "group_size": 32
1762
+ },
1763
+ "model.language_model.layers.44.linear_attn.out_proj": {
1764
+ "quant_algo": "MXFP8",
1765
+ "group_size": 32
1766
+ },
1767
+ "model.language_model.layers.45.linear_attn.in_proj_a": {
1768
+ "quant_algo": "MXFP8",
1769
+ "group_size": 32
1770
+ },
1771
+ "model.language_model.layers.45.linear_attn.in_proj_b": {
1772
+ "quant_algo": "MXFP8",
1773
+ "group_size": 32
1774
+ },
1775
+ "model.language_model.layers.45.linear_attn.in_proj_qkv": {
1776
+ "quant_algo": "MXFP8",
1777
+ "group_size": 32
1778
+ },
1779
+ "model.language_model.layers.45.linear_attn.in_proj_z": {
1780
+ "quant_algo": "MXFP8",
1781
+ "group_size": 32
1782
+ },
1783
+ "model.language_model.layers.45.linear_attn.out_proj": {
1784
+ "quant_algo": "MXFP8",
1785
+ "group_size": 32
1786
+ },
1787
+ "model.language_model.layers.46.linear_attn.in_proj_a": {
1788
+ "quant_algo": "MXFP8",
1789
+ "group_size": 32
1790
+ },
1791
+ "model.language_model.layers.46.linear_attn.in_proj_b": {
1792
+ "quant_algo": "MXFP8",
1793
+ "group_size": 32
1794
+ },
1795
+ "model.language_model.layers.46.linear_attn.in_proj_qkv": {
1796
+ "quant_algo": "MXFP8",
1797
+ "group_size": 32
1798
+ },
1799
+ "model.language_model.layers.46.linear_attn.in_proj_z": {
1800
+ "quant_algo": "MXFP8",
1801
+ "group_size": 32
1802
+ },
1803
+ "model.language_model.layers.46.linear_attn.out_proj": {
1804
+ "quant_algo": "MXFP8",
1805
+ "group_size": 32
1806
+ },
1807
+ "model.language_model.layers.47.self_attn.indexer.index_qk_proj": {
1808
+ "quant_algo": "MXFP8",
1809
+ "group_size": 32
1810
+ },
1811
+ "model.language_model.layers.47.self_attn.k_proj": {
1812
+ "quant_algo": "MXFP8",
1813
+ "group_size": 32
1814
+ },
1815
+ "model.language_model.layers.47.self_attn.o_proj": {
1816
+ "quant_algo": "MXFP8",
1817
+ "group_size": 32
1818
+ },
1819
+ "model.language_model.layers.47.self_attn.q_proj": {
1820
+ "quant_algo": "MXFP8",
1821
+ "group_size": 32
1822
+ },
1823
+ "model.language_model.layers.47.self_attn.v_proj": {
1824
+ "quant_algo": "MXFP8",
1825
+ "group_size": 32
1826
+ },
1827
+ "model.language_model.layers.5.linear_attn.in_proj_a": {
1828
+ "quant_algo": "MXFP8",
1829
+ "group_size": 32
1830
+ },
1831
+ "model.language_model.layers.5.linear_attn.in_proj_b": {
1832
+ "quant_algo": "MXFP8",
1833
+ "group_size": 32
1834
+ },
1835
+ "model.language_model.layers.5.linear_attn.in_proj_qkv": {
1836
+ "quant_algo": "MXFP8",
1837
+ "group_size": 32
1838
+ },
1839
+ "model.language_model.layers.5.linear_attn.in_proj_z": {
1840
+ "quant_algo": "MXFP8",
1841
+ "group_size": 32
1842
+ },
1843
+ "model.language_model.layers.5.linear_attn.out_proj": {
1844
+ "quant_algo": "MXFP8",
1845
+ "group_size": 32
1846
+ },
1847
+ "model.language_model.layers.6.linear_attn.in_proj_a": {
1848
+ "quant_algo": "MXFP8",
1849
+ "group_size": 32
1850
+ },
1851
+ "model.language_model.layers.6.linear_attn.in_proj_b": {
1852
+ "quant_algo": "MXFP8",
1853
+ "group_size": 32
1854
+ },
1855
+ "model.language_model.layers.6.linear_attn.in_proj_qkv": {
1856
+ "quant_algo": "MXFP8",
1857
+ "group_size": 32
1858
+ },
1859
+ "model.language_model.layers.6.linear_attn.in_proj_z": {
1860
+ "quant_algo": "MXFP8",
1861
+ "group_size": 32
1862
+ },
1863
+ "model.language_model.layers.6.linear_attn.out_proj": {
1864
+ "quant_algo": "MXFP8",
1865
+ "group_size": 32
1866
+ },
1867
+ "model.language_model.layers.7.self_attn.indexer.index_qk_proj": {
1868
+ "quant_algo": "MXFP8",
1869
+ "group_size": 32
1870
+ },
1871
+ "model.language_model.layers.7.self_attn.k_proj": {
1872
+ "quant_algo": "MXFP8",
1873
+ "group_size": 32
1874
+ },
1875
+ "model.language_model.layers.7.self_attn.o_proj": {
1876
+ "quant_algo": "MXFP8",
1877
+ "group_size": 32
1878
+ },
1879
+ "model.language_model.layers.7.self_attn.q_proj": {
1880
+ "quant_algo": "MXFP8",
1881
+ "group_size": 32
1882
+ },
1883
+ "model.language_model.layers.7.self_attn.v_proj": {
1884
+ "quant_algo": "MXFP8",
1885
+ "group_size": 32
1886
+ },
1887
+ "model.language_model.layers.8.linear_attn.in_proj_a": {
1888
+ "quant_algo": "MXFP8",
1889
+ "group_size": 32
1890
+ },
1891
+ "model.language_model.layers.8.linear_attn.in_proj_b": {
1892
+ "quant_algo": "MXFP8",
1893
+ "group_size": 32
1894
+ },
1895
+ "model.language_model.layers.8.linear_attn.in_proj_qkv": {
1896
+ "quant_algo": "MXFP8",
1897
+ "group_size": 32
1898
+ },
1899
+ "model.language_model.layers.8.linear_attn.in_proj_z": {
1900
+ "quant_algo": "MXFP8",
1901
+ "group_size": 32
1902
+ },
1903
+ "model.language_model.layers.8.linear_attn.out_proj": {
1904
+ "quant_algo": "MXFP8",
1905
+ "group_size": 32
1906
+ },
1907
+ "model.language_model.layers.9.linear_attn.in_proj_a": {
1908
+ "quant_algo": "MXFP8",
1909
+ "group_size": 32
1910
+ },
1911
+ "model.language_model.layers.9.linear_attn.in_proj_b": {
1912
+ "quant_algo": "MXFP8",
1913
+ "group_size": 32
1914
+ },
1915
+ "model.language_model.layers.9.linear_attn.in_proj_qkv": {
1916
+ "quant_algo": "MXFP8",
1917
+ "group_size": 32
1918
+ },
1919
+ "model.language_model.layers.9.linear_attn.in_proj_z": {
1920
+ "quant_algo": "MXFP8",
1921
+ "group_size": 32
1922
+ },
1923
+ "model.language_model.layers.9.linear_attn.out_proj": {
1924
+ "quant_algo": "MXFP8",
1925
+ "group_size": 32
1926
+ },
1927
+ "mtp.layers.48.mlp.experts": {
1928
+ "quant_algo": "W4A16_NVFP4",
1929
+ "group_size": 16
1930
+ }
1931
+ }
1932
+ },
1933
+ "pb_hybrid_layers": [
1934
+ 3,
1935
+ 7,
1936
+ 11,
1937
+ 15,
1938
+ 19,
1939
+ 23,
1940
+ 27,
1941
+ 31,
1942
+ 35,
1943
+ 39,
1944
+ 43,
1945
+ 47
1946
+ ],
1947
+ "pb_attention_window": 16384,
1948
+ "pb_attention_sinks": 4
1949
+ }