Duplicate from ctech123/Qwen3.8-Flash-Next-E384-FP4-V2

.gitattributes

@@ -0,0 +1,37 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,108 @@
+---
+license: other
+license_name: nvidia-open-model-agreement
+license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/
+base_model: autotrust/Qwen3.8-Flash-Next-E384-FP4-Local
+base_model_relation: merge
+pipeline_tag: image-text-to-text
+library_name: vllm
+language:
+- en
+- zh
+tags:
+- qwen3.8
+- moe
+- nvfp4
+- modelopt
+- mtp
+- rl
+- grpo
+---
+
+# Qwen3.8-Flash-Next-E384-FP4-V2
+
+A compact build of [nvidia/Qwen3.8-Flash-Next-FP4-Local](https://huggingface.co/nvidia/Qwen3.8-Flash-Next-FP4-Local) found by
+**Neural Architecture Search (NAS)** over the routed-expert configuration: it keeps **384 of the 512 routed experts** in every
+decoder MoE layer and in the MTP layer.
+
+Unchanged from the original:
+- top-10 routing and the shared experts;
+- attention, the n-gram PLE table, the vision tower and the tokenizer.
+
+This is an unofficial derivative, not produced or endorsed by Alibaba or NVIDIA.
+
+| | original (512 experts) | **E384** |
+|---|---|---|
+| weights | 102.14 GB = 95.12 GiB | **85.66 GB = 79.78 GiB** (−15.35 GiB, −16.1 %) |
+| routed experts / layer (decoder + MTP) | 512 | 384 |
+| active experts / token | 10 + shared | 10 + shared |
+
+Full merged checkpoint of the E384 FP4 model after GRPO-style reinforcement learning with a LoRA
+adapter. **This is a complete model, not an adapter**: all 21 weight shards, the tokenizer, configs and
+MTP files are included. It is an unofficial derivative and is not endorsed by Alibaba or NVIDIA.
+
+## Serving
+
+Validated with vLLM 0.31 on one B200. The included `vllm_patch/sitecustomize.py` is needed for this
+checkpoint's FP4/CSF/PLE weight layout. From this directory:
+
+```bash
+export PYTHONPATH="$PWD/vllm_patch${PYTHONPATH:+:$PYTHONPATH}"
+vllm serve . --served-model-name e384-rl --port 8000 \
+  --tensor-parallel-size 1 --dtype bfloat16 --kv-cache-dtype fp8 \
+  --quantization modelopt_mixed --mamba-cache-mode align \
+  --enable-prefix-caching --enable-chunked-prefill \
+  --max-model-len 72000 --max-num-seqs 128 --gpu-memory-utilization 0.90 \
+  --reasoning-parser qwen3 --tool-call-parser qwen3_xml --enable-auto-tool-choice
+```
+
+`serve_vllm.sh` is the equivalent portable launcher. The model was benchmarked through this serving
+path, including the FP8 KV cache and the Qwen reasoning/tool parsers.
+
+## Training and checkpoint contents
+
+- Three GRPO-style LoRA updates with verifiable rewards; the final round targets short-budget CVSS
+  completion and structured findings emission.
+- LoRA rank 32, alpha 64. The routed experts, router, shared expert and PLE are frozen.
+- The GDN projection deltas are folded at full precision and those projections are stored as BF16;
+  the corresponding entries are removed from `quantization_config.quantized_layers`. Other model
+  tensors retain the base checkpoint format.
+- The final adapter was merged into the weights before this repository was created. No adapter loading
+  step is needed.
+
+## Full benchmark results
+
+Same benchmark harness and serving settings for all rows. These are single runs; GPQA has only 198
+questions and is correspondingly noisy. AIME 2026 uses 30 problems x 4 samples with xhigh reasoning and
+a 64k generation budget. Base numbers are the previously published reference numbers, not a rerun.
+
+| benchmark | base reference | this model |
+|---|---:|---:|
+| GPQA-Diamond (low, 32k) | 77.3 | **75.76** |
+| MMLU (14,042) | 86.9 | **87.37** |
+| C-Eval val (1,346) | 89.0 | **89.97** |
+| AIME 2026 (xhigh, 64k) | 93.3 | **95.0** |
+| HumanEval (164, greedy) | 97.6 | **98.78** |
+| MMMU val (900) | 79.4 | **78.22** |
+| tool calling (800) | — | **77.88** |
+
+Detailed per-task counts and truncation statistics are in `benchmark_results.json`.
+
+### AIME reasoning length
+
+Compared with the preceding RL checkpoint on the same 120 AIME samples, total completion tokens fell
+2.05% (19,897 to 19,490 per answer on average), while accuracy rose from 92.5 to 95.0. The median was
+essentially unchanged (16,528 to 16,675 tokens); the reduction came mainly from the long tail (P90
+58,272 to 45,339) and fewer 65,536-token truncations (9 to 6). `completion_tokens` includes both the
+reasoning trace and the final answer.
+
+## Behavior notes and limitations
+
+On the same training-prompt evaluation used during the final RL round, CVSS3 completion within a 2,200
+token budget rose from 39% to 70%, CVSS4 from 18% to 60%, and structured-findings emission reward from
+0.609 to 0.917. Math and guard rewards did not degrade in that evaluation. These are training-prompt
+measurements, not an independent behavior benchmark.
+
+**CVSS arithmetic is still not reliable**: among completed answers, exact agreement with the reference
+implementation was about 4.5% for CVSS3 and 4.2% for CVSS4 in that evaluation. This checkpoint should not
+be described as having fully fixed CVSS scoring. See `behavior_results.json` for the detailed comparison.

behavior_results.json

@@ -0,0 +1,135 @@
+{
+ "description": "Iteration-2 short-budget CVSS RL round; same 351 training prompts x 8 samples for both evaluations, temperature 1.0/top_p 0.95. iter2 file = it1 policy before the update; iter3 file = it2 policy after the update. Not held-out.",
+ "changes": {
+  "cvss_max_tokens": 2200,
+  "reward": "lenient score extraction + continuous absolute-error reward + exact bonus + cvss3 severity credit",
+  "training": "group by full (task,prompt), skip empty CVSS truncations from backpropagation",
+  "warm_start": "grpo/lora_it1/ckpt"
+ },
+ "it1_policy": {
+  "tasks": {
+   "cvss3": {
+    "samples": 880,
+    "mean_reward": 0.1678,
+    "avg_completion_tokens": 1998
+   },
+   "cvss4": {
+    "samples": 440,
+    "mean_reward": 0.0545,
+    "avg_completion_tokens": 2167
+   },
+   "emission": {
+    "samples": 360,
+    "mean_reward": 0.6089,
+    "avg_completion_tokens": 973
+   },
+   "format": {
+    "samples": 240,
+    "mean_reward": 1.0,
+    "avg_completion_tokens": 36
+   },
+   "guard": {
+    "samples": 128,
+    "mean_reward": 0.8047,
+    "avg_completion_tokens": 761
+   },
+   "math": {
+    "samples": 720,
+    "mean_reward": 0.7542,
+    "avg_completion_tokens": 333
+   },
+   "retire": {
+    "samples": 40,
+    "mean_reward": 0.66,
+    "avg_completion_tokens": 1040
+   }
+  },
+  "cvss_detail": {
+   "cvss3": {
+    "samples": 880,
+    "finished": 344,
+    "parsed": 279,
+    "exact": 16,
+    "within_0.3": 36,
+    "fake_zero": 31,
+    "exact_over_all": 0.0182,
+    "exact_over_finished": 0.0465,
+    "within_0.3_over_finished": 0.1047
+   },
+   "cvss4": {
+    "samples": 440,
+    "finished": 77,
+    "parsed": 39,
+    "exact": 2,
+    "within_0.3": 9,
+    "fake_zero": 0,
+    "exact_over_all": 0.0045,
+    "exact_over_finished": 0.026,
+    "within_0.3_over_finished": 0.1169
+   }
+  }
+ },
+ "it2_policy": {
+  "tasks": {
+   "cvss3": {
+    "samples": 880,
+    "mean_reward": 0.3527,
+    "avg_completion_tokens": 1783
+   },
+   "cvss4": {
+    "samples": 440,
+    "mean_reward": 0.2948,
+    "avg_completion_tokens": 1991
+   },
+   "emission": {
+    "samples": 360,
+    "mean_reward": 0.9169,
+    "avg_completion_tokens": 838
+   },
+   "format": {
+    "samples": 240,
+    "mean_reward": 1.0,
+    "avg_completion_tokens": 35
+   },
+   "guard": {
+    "samples": 128,
+    "mean_reward": 0.8125,
+    "avg_completion_tokens": 876
+   },
+   "math": {
+    "samples": 720,
+    "mean_reward": 0.8347,
+    "avg_completion_tokens": 348
+   },
+   "retire": {
+    "samples": 40,
+    "mean_reward": 0.64,
+    "avg_completion_tokens": 1019
+   }
+  },
+  "cvss_detail": {
+   "cvss3": {
+    "samples": 880,
+    "finished": 617,
+    "parsed": 553,
+    "exact": 28,
+    "within_0.3": 74,
+    "fake_zero": 26,
+    "exact_over_all": 0.0318,
+    "exact_over_finished": 0.0454,
+    "within_0.3_over_finished": 0.1199
+   },
+   "cvss4": {
+    "samples": 440,
+    "finished": 264,
+    "parsed": 201,
+    "exact": 11,
+    "within_0.3": 39,
+    "fake_zero": 1,
+    "exact_over_all": 0.025,
+    "exact_over_finished": 0.0417,
+    "within_0.3_over_finished": 0.1477
+   }
+  }
+ }
+}

benchmark_results.json

@@ -0,0 +1,73 @@
+{
+ "mmmu": {
+  "n": 900,
+  "ok": 704,
+  "trunc": 1,
+  "errors": 0,
+  "tokens": 1044906,
+  "acc": 78.22,
+  "total": 900,
+  "minutes": 11.3
+ },
+ "toolcall": {
+  "n": 800,
+  "ok": 623,
+  "trunc": 1,
+  "errors": 0,
+  "tokens": 149727,
+  "name_ok": 667,
+  "acc": 77.88,
+  "total": 800,
+  "minutes": 12.4
+ },
+ "gpqa": {
+  "n": 198,
+  "ok": 150,
+  "trunc": 3,
+  "errors": 0,
+  "tokens": 812122,
+  "acc": 75.76,
+  "total": 198,
+  "minutes": 25.8
+ },
+ "ceval": {
+  "n": 1346,
+  "ok": 1211,
+  "trunc": 0,
+  "errors": 0,
+  "tokens": 917558,
+  "acc": 89.97,
+  "total": 1346,
+  "minutes": 26.9
+ },
+ "aime26": {
+  "n": 120,
+  "ok": 114,
+  "trunc": 6,
+  "errors": 0,
+  "tokens": 2338742,
+  "acc": 95.0,
+  "total": 120,
+  "minutes": 54.1
+ },
+ "humaneval": {
+  "n": 164,
+  "ok": 162,
+  "trunc": 0,
+  "errors": 0,
+  "tokens": 110345,
+  "acc": 98.78,
+  "total": 164,
+  "minutes": 79.3
+ },
+ "mmlu": {
+  "n": 14042,
+  "ok": 12269,
+  "trunc": 8,
+  "errors": 0,
+  "tokens": 9486427,
+  "acc": 87.37,
+  "total": 14042,
+  "minutes": 81.0
+ }
+}

chat_template.jinja

@@ -0,0 +1,170 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set reasoning_instructions = '' %}
+{%- if enable_thinking is undefined or enable_thinking is true %}
+    {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+    {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}
+        {{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), medium, and low.') }}
+    {%- endif %}
+    {%- if resolved_reasoning_effort == 'xhigh' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
+    {%- elif resolved_reasoning_effort == 'low' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
+    {%- endif %}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {%- if reasoning_instructions %}
+        {{- reasoning_instructions + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '')  + content + '<|im_end|>\n' }}
+        {%- elif reasoning_instructions %}
+            {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+        {%- endif %}
+    {%- elif reasoning_instructions %}
+        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if preserve_thinking is undefined or preserve_thinking is true or loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined and tool_call.arguments != '' %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,1682 @@
+{
+  "architectures": [
+    "Qwen4ExpForConditionalGeneration"
+  ],
+  "image_token_id": 248056,
+  "language_model_only": false,
+  "model_type": "qwen4_exp",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 248044,
+    "dtype": "bfloat16",
+    "eos_token_id": 248044,
+    "full_attention_interval": 4,
+    "hc_count": 4,
+    "hc_lowrank": 320,
+    "head_dim": 256,
+    "heads_per_ngram": 8,
+    "hidden_act": "silu",
+    "hidden_size": 2560,
+    "indexer_budget": 2048,
+    "indexer_compress_ratio": 4,
+    "indexer_head_dim": 128,
+    "indexer_kv_heads": 1,
+    "indexer_n_heads": 4,
+    "initializer_range": 0.02,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention"
+    ],
+    "linear_conv_kernel_dim": 4,
+    "linear_key_head_dim": 128,
+    "linear_num_key_heads": 16,
+    "linear_num_value_heads": 48,
+    "linear_value_head_dim": 128,
+    "make_ngram_vocab_size_divisible_by": 128,
+    "mamba_ssm_dtype": "float32",
+    "max_position_embeddings": 262144,
+    "model_type": "qwen4_exp_text",
+    "moe_intermediate_size": 640,
+    "mtp": {
+      "hybrid": true,
+      "layer_types": [
+        "full_attention"
+      ],
+      "mtp_use_hidden_state_from_layer": null,
+      "num_hidden_layers": 1,
+      "rope_theta": 10000000
+    },
+    "mtp_num_hidden_layers": 1,
+    "mtp_use_dedicated_embeddings": false,
+    "ngram_size": 3,
+    "ngram_vocab_size_base": 20000000,
+    "num_attention_heads": 24,
+    "num_experts": 384,
+    "num_experts_per_tok": 10,
+    "num_hidden_layers": 48,
+    "num_key_value_heads": 2,
+    "output_gate_type": "sigmoid",
+    "output_router_logits": false,
+    "pad_token_id": null,
+    "partial_rotary_factor": 0.25,
+    "ple_conv_kernel_size": 4,
+    "ple_embed_dim": 2560,
+    "ple_layer_ids": [
+      2
+    ],
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        11,
+        11,
+        10
+      ],
+      "partial_rotary_factor": 0.25,
+      "rope_theta": 10000000,
+      "rope_type": "default"
+    },
+    "router_aux_loss_coef": 0.001,
+    "shared_expert_intermediate_size": 640,
+    "split_ngram_parts": 128,
+    "tie_word_embeddings": false,
+    "use_cache": true,
+    "vocab_size": 248320,
+    "ple_embedding_dtype": "nvfp4"
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.8.0.dev0",
+  "video_token_id": 248057,
+  "vision_config": {
+    "deepstack_visual_indexes": [],
+    "depth": 27,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4304,
+    "model_type": "qwen4_exp",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2560,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053,
+  "quantization_config": {
+    "quant_method": "modelopt",
+    "quant_algo": "MIXED_PRECISION",
+    "kv_cache_quant_algo": null,
+    "ignore": [
+      "lm_head",
+      "mtp.fc_embedding",
+      "mtp.fc_hidden",
+      "mtp.pre_fc_norm_*",
+      "mtp.hyper_connection_mixer*",
+      "mtp.layers.0.self_attn*",
+      "mtp.layers.0.attn_hyper_connection*",
+      "mtp.layers.0.mlp_hyper_connection*",
+      "mtp.layers.0.mlp.gate",
+      "mtp.layers.0.mlp.shared_expert*"
+    ],
+    "quantized_layers": {
+      "model.language_model.layers.0.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.1.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.2.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.3.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.4.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.5.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.6.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.7.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.8.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.9.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.10.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.11.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.12.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.13.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.14.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.15.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.16.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.17.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.18.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.19.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.20.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.21.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.22.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.23.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.24.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.25.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.26.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.27.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.28.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.29.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.30.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.31.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.32.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.33.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.34.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.35.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.36.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.37.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.38.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.39.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.40.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.41.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.42.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.43.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.44.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.45.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.46.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.47.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16,
+        "weight_scale_encoding": "csf"
+      },
+      "model.language_model.layers.0.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.0.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.0.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.1.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.1.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.1.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.2.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.2.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.2.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.3.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.3.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.3.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.4.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.4.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.4.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.5.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.5.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.5.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.6.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.6.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.6.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.7.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.7.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.7.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.8.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.8.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.8.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.9.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.9.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.9.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.10.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.10.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.10.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.11.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.11.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.11.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.12.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.12.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.12.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.13.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.13.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.13.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.14.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.14.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.14.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.15.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.15.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.15.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.16.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.16.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.16.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.17.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.17.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.17.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.18.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.18.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.18.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.19.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.19.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.19.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.20.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.20.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.20.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.21.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.21.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.21.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.22.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.22.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.22.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.23.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.23.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.23.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.24.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.24.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.24.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.25.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.25.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.25.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.26.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.26.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.26.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.27.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.27.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.27.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.28.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.28.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.28.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.29.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.29.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.29.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.30.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.30.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.30.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.31.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.31.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.31.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.32.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.32.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.32.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.33.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.33.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.33.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.34.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.34.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.34.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.35.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.35.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.35.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.36.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.36.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.36.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.37.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.37.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.37.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.38.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.38.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.38.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.39.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.39.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.39.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.40.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.40.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.40.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.41.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.41.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.41.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.42.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.42.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.42.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.43.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.43.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.43.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.44.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.44.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.44.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.45.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.45.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.45.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.46.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.46.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.46.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.47.mlp.shared_expert.gate_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.47.mlp.shared_expert.up_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.47.mlp.shared_expert.down_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.1.ple.ple_embedding.ngram_embedding": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "mtp.layers.0.mlp.experts": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.language_model.layers.11.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.11.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.11.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.11.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.11.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.15.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.15.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.15.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.15.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.15.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.19.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.19.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.19.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.19.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.19.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.23.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.23.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.23.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.23.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.23.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.27.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.27.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.27.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.27.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.27.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.3.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.3.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.3.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.3.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.3.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.31.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.31.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.31.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.31.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.31.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.35.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.35.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.35.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.35.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.35.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.39.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.39.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.39.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.39.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.39.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.43.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.43.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.43.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.43.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.43.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.47.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.47.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.47.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.47.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.47.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.7.self_attn.indexer.index_qk_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.7.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.7.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.7.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.language_model.layers.7.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "mtp.layers.48.mlp.experts": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.0.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.0.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.1.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.1.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.10.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.10.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.11.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.11.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.12.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.12.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.13.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.13.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.14.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.14.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.15.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.15.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.16.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.16.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.17.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.17.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.18.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.18.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.19.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.19.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.2.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.2.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.20.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.20.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.21.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.21.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.22.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.22.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.23.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.23.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.24.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.24.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.25.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.25.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.26.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.26.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.3.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.3.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.4.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.4.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.5.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.5.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.6.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.6.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.7.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.7.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.8.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.8.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.9.attn.proj": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.9.attn.qkv": {
+        "quant_algo": "MXFP8",
+        "group_size": 32
+      },
+      "model.visual.blocks.0.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.0.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.1.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.1.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.10.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.10.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.11.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.11.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.12.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.12.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.13.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.13.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.14.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.14.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.15.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.15.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.16.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.16.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.17.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.17.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.18.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.18.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.19.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.19.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.2.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.2.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.20.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.20.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.21.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.21.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.22.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.22.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.23.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.23.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.24.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.24.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.25.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.25.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.26.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.26.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.3.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.3.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.4.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.4.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.5.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.5.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.6.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.6.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.7.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.7.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.8.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.8.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.9.mlp.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.blocks.9.mlp.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.merger.linear_fc1": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      },
+      "model.visual.merger.linear_fc2": {
+        "quant_algo": "W4A16_NVFP4",
+        "group_size": 16
+      }
+    }
+  }
+}

expert_selection.json

@@ -0,0 +1 @@
+{"keep": 384, "metric": "acp", "normalize": true, "files_with_weights": [["harvest_s1.pt", 1.0], ["harvest_s2.pt", 1.0], ["science_s5.pt", 2.0], ["zh_s6.pt", 3.0], ["code_s7.pt", 2.0], ["agent_s8.pt", 2.0], ["math_s9.pt", 3.0], ["code_glm_s11.pt", 2.0], ["agent_glm_s12.pt", 3.0], ["swe_agent_s13.pt", 3.0], ["olympiad_glm_s14.pt", 2.0], ["en_web_s15.pt", 2.0], ["zh_web_s16.pt", 3.0], ["vision_s17.pt", 3.0]], "experts": {"0": [0, 1, 4, 5, 7, 9, 10, 12, 14, 15, 16, 17, 19, 20, 21, 23, 24, 26, 27, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 42, 44, 45, 46, 47, 48, 49, 51, 52, 53, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 76, 79, 80, 81, 83, 84, 87, 88, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 103, 105, 107, 108, 109, 110, 111, 114, 116, 119, 120, 121, 122, 123, 124, 125, 126, 128, 129, 130, 131, 132, 135, 138, 139, 140, 142, 143, 144, 145, 147, 148, 149, 150, 151, 153, 154, 155, 156, 157, 159, 160, 162, 164, 165, 166, 167, 168, 169, 172, 173, 175, 176, 178, 179, 180, 182, 183, 184, 186, 187, 188, 190, 192, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 208, 209, 211, 212, 213, 214, 215, 217, 218, 219, 220, 222, 223, 224, 225, 226, 227, 228, 230, 231, 232, 233, 234, 235, 239, 240, 241, 242, 243, 246, 247, 248, 249, 251, 253, 255, 256, 258, 259, 261, 262, 264, 265, 266, 267, 268, 269, 270, 272, 273, 274, 275, 277, 278, 280, 282, 283, 285, 287, 288, 289, 290, 291, 293, 294, 297, 298, 299, 301, 302, 303, 304, 305, 306, 308, 309, 310, 312, 313, 314, 315, 316, 319, 320, 321, 322, 325, 326, 328, 329, 330, 332, 333, 334, 337, 338, 340, 342, 343, 344, 346, 347, 348, 349, 351, 352, 353, 354, 356, 358, 359, 360, 361, 362, 363, 364, 365, 367, 369, 370, 371, 372, 374, 376, 377, 378, 379, 380, 381, 383, 384, 385, 386, 388, 389, 390, 391, 392, 393, 394, 395, 396, 398, 399, 400, 401, 402, 403, 404, 405, 406, 408, 411, 412, 415, 416, 417, 418, 419, 423, 425, 426, 427, 428, 430, 433, 434, 435, 436, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 453, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 468, 470, 471, 473, 475, 476, 478, 479, 480, 481, 482, 483, 484, 485, 487, 488, 489, 491, 492, 493, 494, 495, 496, 497, 498, 499, 501, 503, 504, 505, 506, 507, 508, 509], "1": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 28, 29, 30, 32, 33, 34, 36, 38, 39, 40, 42, 43, 44, 45, 46, 47, 48, 51, 52, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 78, 80, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 94, 96, 97, 99, 100, 101, 102, 103, 104, 105, 106, 108, 109, 110, 111, 112, 113, 114, 115, 119, 120, 123, 124, 125, 128, 129, 130, 131, 132, 133, 134, 135, 137, 138, 142, 143, 144, 146, 147, 148, 149, 152, 155, 156, 157, 158, 159, 160, 161, 162, 164, 165, 166, 169, 171, 172, 173, 174, 175, 176, 177, 178, 179, 182, 183, 185, 186, 187, 189, 190, 193, 194, 195, 196, 197, 200, 202, 203, 204, 205, 206, 209, 210, 211, 212, 213, 214, 215, 219, 220, 222, 223, 224, 225, 226, 227, 228, 229, 230, 232, 233, 234, 235, 236, 237, 238, 240, 241, 242, 243, 244, 245, 246, 247, 249, 250, 251, 252, 254, 255, 257, 258, 259, 261, 262, 263, 264, 265, 266, 267, 268, 269, 271, 273, 274, 276, 278, 279, 280, 281, 282, 284, 285, 287, 290, 291, 292, 293, 294, 295, 296, 297, 298, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 311, 313, 314, 315, 316, 317, 318, 319, 320, 321, 323, 324, 325, 326, 328, 329, 330, 331, 334, 336, 338, 339, 340, 341, 342, 344, 345, 349, 350, 351, 352, 353, 354, 355, 357, 358, 361, 362, 363, 364, 366, 368, 370, 372, 373, 375, 376, 378, 381, 382, 383, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 400, 401, 402, 403, 404, 405, 406, 407, 409, 410, 413, 414, 415, 417, 418, 420, 422, 423, 426, 427, 429, 431, 432, 434, 435, 437, 438, 440, 441, 443, 444, 445, 446, 447, 449, 450, 451, 453, 454, 456, 457, 459, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 474, 475, 476, 477, 481, 482, 483, 486, 487, 488, 489, 491, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 508, 510], "2": [0, 2, 4, 5, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 20, 21, 22, 23, 25, 27, 28, 29, 30, 32, 34, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 47, 49, 50, 51, 53, 54, 55, 57, 58, 59, 61, 63, 64, 65, 66, 68, 69, 70, 71, 72, 74, 75, 76, 77, 79, 81, 83, 84, 85, 86, 87, 88, 89, 90, 92, 95, 96, 97, 99, 100, 101, 102, 103, 104, 105, 106, 107, 109, 110, 112, 113, 114, 115, 116, 117, 122, 123, 124, 125, 127, 128, 129, 130, 132, 133, 134, 135, 137, 138, 139, 140, 142, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 160, 161, 162, 163, 164, 167, 168, 169, 170, 171, 173, 175, 176, 177, 178, 179, 180, 181, 185, 186, 187, 188, 190, 191, 192, 194, 195, 196, 197, 198, 200, 201, 202, 206, 207, 208, 210, 211, 212, 213, 214, 215, 217, 220, 221, 222, 224, 225, 226, 228, 229, 230, 232, 233, 236, 237, 238, 239, 240, 243, 244, 245, 249, 250, 251, 252, 253, 256, 257, 258, 259, 260, 262, 263, 264, 265, 267, 268, 270, 271, 273, 276, 277, 279, 280, 283, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 300, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 315, 316, 317, 318, 319, 320, 321, 322, 324, 329, 332, 333, 335, 336, 337, 339, 341, 342, 343, 346, 347, 348, 349, 350, 351, 352, 353, 355, 357, 358, 359, 360, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 377, 378, 379, 380, 381, 382, 383, 384, 385, 387, 388, 391, 393, 394, 395, 396, 397, 399, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 424, 425, 426, 427, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 442, 443, 444, 445, 446, 449, 450, 451, 452, 453, 454, 456, 457, 458, 460, 462, 463, 464, 466, 467, 468, 469, 470, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 483, 484, 485, 486, 487, 490, 491, 493, 494, 495, 496, 497, 500, 502, 503, 504, 507, 508, 510, 511], "3": [0, 1, 3, 4, 6, 8, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 23, 24, 25, 26, 27, 29, 30, 31, 32, 33, 35, 36, 37, 39, 40, 44, 45, 47, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 61, 62, 63, 64, 65, 67, 68, 69, 70, 72, 73, 75, 76, 77, 78, 79, 80, 82, 83, 84, 85, 86, 88, 89, 90, 91, 93, 94, 95, 96, 100, 102, 103, 104, 106, 107, 108, 109, 113, 114, 115, 116, 117, 118, 119, 120, 122, 124, 126, 127, 128, 129, 130, 132, 133, 134, 135, 138, 139, 140, 141, 142, 145, 146, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 166, 168, 169, 171, 172, 173, 174, 175, 176, 177, 178, 179, 181, 182, 183, 184, 185, 186, 187, 188, 189, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 202, 203, 204, 205, 207, 208, 210, 211, 212, 213, 214, 215, 216, 217, 219, 220, 221, 223, 225, 226, 228, 229, 230, 231, 233, 235, 236, 237, 238, 239, 240, 241, 242, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 256, 258, 259, 260, 261, 262, 263, 264, 265, 267, 268, 269, 271, 272, 273, 274, 275, 276, 278, 279, 280, 282, 283, 286, 287, 288, 289, 290, 291, 292, 293, 295, 298, 300, 301, 302, 305, 306, 308, 309, 310, 316, 317, 318, 321, 322, 323, 324, 325, 326, 328, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 342, 343, 348, 349, 350, 351, 352, 354, 355, 356, 357, 358, 359, 361, 362, 363, 364, 366, 367, 368, 370, 371, 372, 373, 374, 375, 376, 379, 381, 382, 383, 384, 386, 390, 391, 392, 394, 395, 397, 398, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 415, 416, 418, 419, 420, 421, 422, 426, 428, 429, 430, 432, 433, 434, 435, 436, 437, 438, 440, 441, 442, 443, 444, 446, 447, 450, 451, 452, 454, 455, 456, 457, 459, 460, 461, 465, 466, 468, 469, 470, 471, 473, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 489, 490, 491, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 507, 508, 510], "4": [1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 14, 16, 19, 20, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 48, 49, 50, 51, 53, 54, 55, 56, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 73, 74, 77, 81, 82, 83, 84, 85, 86, 87, 88, 90, 91, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 104, 105, 106, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 120, 121, 122, 123, 125, 126, 128, 130, 131, 132, 133, 135, 136, 138, 139, 141, 143, 146, 147, 148, 149, 150, 152, 153, 155, 156, 158, 159, 160, 161, 162, 163, 164, 166, 167, 168, 169, 170, 171, 173, 174, 175, 176, 178, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 191, 192, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 220, 222, 223, 226, 227, 230, 231, 233, 234, 236, 237, 238, 239, 240, 241, 242, 243, 245, 246, 247, 248, 249, 251, 252, 254, 255, 257, 259, 260, 263, 264, 265, 266, 267, 269, 270, 271, 272, 273, 274, 276, 277, 278, 280, 281, 283, 284, 285, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 298, 299, 300, 302, 303, 306, 307, 308, 309, 310, 311, 312, 313, 314, 316, 318, 319, 321, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 338, 339, 340, 342, 344, 345, 346, 347, 349, 350, 351, 352, 357, 358, 359, 360, 361, 362, 363, 364, 367, 368, 370, 371, 372, 373, 375, 376, 377, 379, 380, 382, 383, 384, 385, 386, 388, 389, 390, 391, 392, 393, 394, 395, 398, 399, 400, 402, 403, 404, 405, 406, 407, 410, 412, 413, 414, 416, 417, 419, 420, 421, 422, 423, 424, 425, 426, 428, 429, 430, 431, 432, 433, 436, 437, 438, 440, 441, 445, 448, 449, 451, 452, 453, 454, 455, 458, 463, 464, 465, 466, 467, 468, 469, 471, 472, 474, 479, 480, 482, 483, 485, 486, 487, 488, 489, 492, 493, 494, 495, 496, 497, 499, 501, 502, 503, 505, 506, 507, 508, 509, 511], "5": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 21, 22, 24, 25, 26, 27, 29, 32, 33, 34, 35, 36, 37, 38, 39, 41, 42, 43, 44, 45, 46, 47, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 63, 65, 67, 68, 71, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 87, 88, 90, 91, 93, 94, 95, 96, 97, 98, 99, 100, 101, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 116, 117, 118, 119, 120, 121, 123, 125, 126, 127, 129, 130, 131, 132, 133, 134, 135, 136, 137, 139, 140, 141, 142, 143, 144, 145, 146, 147, 149, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 163, 164, 165, 166, 167, 169, 170, 171, 172, 174, 176, 177, 178, 179, 180, 181, 185, 186, 187, 188, 190, 191, 192, 194, 195, 196, 197, 201, 202, 203, 204, 206, 208, 209, 210, 211, 212, 213, 214, 215, 217, 218, 219, 220, 222, 224, 225, 226, 228, 229, 233, 234, 236, 237, 238, 239, 240, 241, 242, 244, 245, 247, 250, 252, 253, 254, 255, 257, 258, 259, 260, 261, 262, 263, 264, 266, 268, 270, 273, 274, 275, 277, 278, 279, 280, 283, 284, 285, 286, 287, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 306, 307, 308, 310, 311, 312, 314, 316, 317, 318, 319, 320, 322, 324, 326, 328, 329, 331, 338, 339, 340, 341, 342, 343, 344, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 357, 358, 359, 360, 364, 367, 368, 369, 370, 373, 374, 375, 376, 377, 379, 380, 381, 382, 383, 385, 387, 388, 389, 390, 391, 392, 393, 394, 395, 397, 399, 400, 401, 402, 403, 405, 406, 409, 412, 413, 414, 415, 416, 417, 419, 420, 421, 424, 425, 426, 427, 430, 433, 434, 435, 436, 437, 438, 439, 441, 442, 445, 446, 447, 449, 450, 454, 455, 456, 457, 458, 459, 460, 461, 465, 467, 468, 469, 470, 471, 473, 475, 476, 477, 478, 479, 481, 482, 483, 484, 485, 486, 487, 489, 491, 492, 493, 495, 496, 497, 499, 500, 501, 503, 505, 506, 507, 508, 509, 510], "6": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 15, 16, 17, 18, 21, 22, 24, 25, 26, 27, 30, 32, 33, 36, 37, 39, 40, 42, 43, 45, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 63, 66, 67, 68, 69, 71, 73, 74, 77, 78, 79, 80, 81, 84, 85, 86, 87, 90, 92, 93, 95, 97, 98, 99, 100, 102, 106, 108, 110, 111, 112, 113, 114, 115, 116, 118, 119, 120, 121, 122, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 136, 137, 138, 139, 140, 142, 143, 144, 145, 146, 147, 149, 151, 152, 153, 154, 155, 157, 158, 159, 161, 162, 164, 165, 166, 167, 168, 169, 170, 173, 174, 175, 176, 177, 178, 179, 180, 182, 183, 185, 186, 187, 188, 189, 191, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 204, 205, 207, 208, 209, 210, 212, 213, 214, 215, 216, 217, 219, 220, 221, 222, 223, 224, 227, 228, 229, 230, 231, 233, 234, 236, 237, 238, 239, 240, 241, 243, 245, 247, 248, 249, 250, 251, 252, 253, 255, 256, 258, 259, 260, 262, 263, 264, 265, 266, 267, 269, 271, 272, 275, 276, 279, 280, 281, 282, 283, 284, 285, 286, 288, 289, 291, 292, 295, 296, 303, 304, 305, 306, 307, 308, 309, 310, 312, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 328, 329, 330, 331, 332, 333, 335, 336, 337, 338, 340, 341, 342, 343, 345, 347, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 361, 362, 363, 364, 365, 368, 369, 371, 372, 373, 374, 377, 378, 379, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 401, 402, 403, 404, 407, 408, 409, 412, 414, 415, 416, 417, 419, 421, 422, 423, 424, 427, 428, 429, 431, 433, 434, 435, 436, 437, 438, 441, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 454, 455, 456, 457, 458, 459, 461, 462, 463, 464, 465, 466, 468, 469, 470, 471, 472, 474, 476, 478, 479, 480, 481, 482, 484, 485, 486, 488, 489, 490, 491, 493, 495, 496, 498, 499, 500, 501, 503, 505, 507, 509, 510, 511], "7": [0, 2, 3, 4, 5, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 24, 26, 27, 28, 29, 31, 33, 34, 35, 36, 37, 38, 39, 41, 43, 44, 46, 47, 48, 49, 51, 52, 53, 54, 55, 56, 58, 60, 61, 62, 63, 64, 65, 66, 67, 68, 70, 71, 72, 73, 74, 75, 76, 77, 80, 81, 82, 84, 85, 86, 92, 93, 94, 95, 96, 97, 101, 102, 103, 104, 105, 107, 108, 109, 112, 113, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 132, 134, 135, 136, 137, 139, 141, 142, 143, 144, 145, 147, 148, 151, 152, 154, 155, 156, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 170, 171, 172, 173, 174, 175, 176, 177, 179, 181, 182, 184, 185, 186, 187, 188, 190, 191, 193, 194, 195, 199, 200, 201, 202, 204, 206, 208, 209, 212, 213, 215, 217, 219, 220, 221, 223, 224, 225, 226, 227, 228, 230, 231, 232, 234, 235, 237, 238, 239, 240, 241, 242, 244, 245, 250, 251, 252, 253, 255, 256, 257, 258, 259, 260, 261, 262, 263, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 279, 281, 282, 283, 285, 287, 288, 290, 291, 292, 294, 295, 296, 298, 299, 301, 302, 303, 304, 305, 306, 307, 308, 309, 311, 312, 315, 316, 317, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 331, 332, 333, 334, 338, 339, 341, 343, 344, 345, 346, 348, 349, 350, 351, 352, 353, 354, 355, 357, 360, 363, 364, 365, 367, 368, 369, 371, 372, 374, 375, 376, 378, 381, 382, 383, 384, 385, 386, 387, 389, 390, 391, 392, 394, 395, 396, 397, 398, 399, 400, 401, 402, 404, 405, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 422, 423, 424, 425, 426, 429, 430, 432, 435, 436, 437, 438, 439, 441, 442, 443, 444, 445, 448, 449, 450, 451, 452, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 469, 470, 471, 472, 473, 474, 477, 479, 480, 481, 482, 485, 486, 489, 490, 491, 492, 493, 494, 495, 496, 498, 499, 500, 501, 502, 503, 505, 506, 507, 508, 509, 510], "8": [1, 2, 3, 4, 6, 7, 9, 10, 11, 13, 14, 15, 16, 20, 21, 23, 24, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 37, 39, 40, 41, 43, 44, 46, 48, 49, 50, 51, 52, 53, 54, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 70, 73, 74, 75, 76, 77, 78, 80, 81, 83, 85, 86, 87, 88, 90, 92, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 108, 109, 110, 111, 113, 115, 116, 119, 120, 121, 122, 124, 125, 126, 127, 132, 133, 135, 136, 138, 139, 140, 141, 143, 144, 145, 146, 151, 154, 155, 157, 158, 159, 160, 161, 162, 163, 164, 166, 167, 169, 170, 171, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 186, 187, 189, 191, 193, 195, 197, 198, 199, 201, 202, 204, 206, 207, 208, 209, 210, 211, 212, 213, 215, 216, 217, 218, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 241, 242, 244, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 258, 260, 261, 262, 266, 267, 268, 270, 271, 272, 273, 274, 275, 276, 277, 280, 281, 283, 286, 287, 289, 290, 292, 293, 294, 296, 297, 299, 301, 302, 303, 304, 305, 307, 308, 309, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 328, 329, 330, 331, 333, 334, 335, 337, 338, 339, 340, 341, 343, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 368, 369, 370, 371, 373, 374, 375, 376, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 391, 394, 395, 397, 398, 399, 402, 403, 404, 405, 408, 410, 411, 412, 414, 415, 416, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 429, 431, 433, 434, 437, 439, 442, 443, 444, 446, 447, 448, 449, 450, 451, 455, 456, 457, 458, 461, 462, 465, 467, 468, 469, 470, 471, 477, 478, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 493, 494, 495, 496, 497, 498, 499, 500, 501, 505, 507, 508, 509, 510, 511], "9": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 42, 44, 45, 46, 48, 49, 50, 51, 52, 54, 55, 56, 58, 59, 61, 62, 63, 64, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 83, 84, 86, 88, 89, 90, 92, 94, 95, 96, 100, 101, 102, 103, 107, 109, 111, 113, 115, 117, 118, 120, 122, 124, 126, 127, 128, 129, 130, 132, 133, 135, 137, 138, 139, 140, 141, 142, 144, 145, 146, 147, 148, 149, 151, 152, 153, 154, 155, 157, 158, 159, 160, 161, 162, 163, 165, 166, 167, 168, 169, 171, 172, 173, 174, 175, 176, 178, 180, 182, 183, 184, 185, 186, 187, 189, 190, 191, 192, 194, 195, 200, 201, 203, 204, 205, 206, 208, 210, 211, 212, 213, 214, 215, 217, 219, 221, 222, 223, 224, 225, 227, 228, 230, 232, 235, 237, 238, 239, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 266, 267, 268, 269, 271, 272, 273, 274, 279, 281, 282, 283, 284, 285, 286, 287, 288, 293, 294, 295, 299, 300, 301, 302, 304, 305, 306, 307, 309, 312, 313, 316, 317, 318, 319, 320, 321, 325, 326, 327, 328, 329, 332, 333, 334, 335, 336, 337, 339, 340, 341, 342, 343, 346, 347, 348, 349, 352, 354, 355, 356, 357, 358, 359, 360, 361, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 376, 377, 378, 381, 382, 385, 388, 389, 390, 391, 392, 393, 395, 396, 397, 398, 399, 400, 403, 404, 405, 406, 408, 409, 410, 411, 412, 414, 415, 416, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 431, 432, 433, 434, 435, 436, 438, 439, 440, 442, 443, 444, 446, 449, 451, 453, 454, 455, 456, 457, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 470, 471, 472, 473, 474, 475, 476, 477, 478, 481, 483, 484, 486, 487, 488, 490, 491, 493, 494, 495, 496, 497, 498, 499, 500, 502, 503, 505, 506, 507, 508, 509, 510, 511], "10": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 23, 24, 26, 27, 28, 29, 30, 31, 32, 33, 35, 37, 38, 41, 42, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 59, 60, 61, 63, 64, 65, 66, 68, 69, 71, 73, 75, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 91, 92, 93, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 115, 117, 119, 121, 122, 123, 125, 126, 129, 130, 131, 132, 133, 136, 137, 138, 139, 141, 142, 143, 146, 147, 148, 151, 152, 153, 154, 155, 156, 157, 158, 160, 161, 162, 163, 164, 165, 166, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 186, 188, 189, 190, 191, 193, 194, 195, 196, 197, 198, 199, 202, 203, 205, 206, 207, 208, 209, 211, 212, 213, 214, 215, 216, 217, 218, 220, 221, 222, 223, 227, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 251, 252, 253, 254, 257, 258, 260, 263, 264, 265, 266, 267, 268, 269, 270, 272, 273, 274, 275, 276, 278, 279, 280, 282, 283, 285, 290, 291, 292, 293, 294, 295, 296, 298, 299, 301, 302, 303, 305, 307, 308, 310, 311, 313, 314, 315, 317, 318, 319, 320, 322, 323, 324, 325, 326, 327, 328, 330, 331, 332, 333, 335, 336, 339, 340, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 358, 361, 362, 364, 365, 366, 367, 369, 370, 371, 373, 374, 375, 376, 377, 378, 379, 380, 381, 384, 386, 389, 391, 393, 396, 397, 398, 399, 400, 401, 402, 403, 406, 409, 410, 411, 413, 415, 419, 421, 422, 424, 425, 426, 427, 430, 431, 432, 434, 435, 437, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 453, 454, 457, 458, 459, 460, 461, 462, 463, 464, 465, 467, 468, 469, 472, 473, 474, 476, 477, 480, 482, 483, 484, 485, 487, 488, 489, 490, 492, 493, 494, 496, 497, 498, 499, 501, 502, 503, 504, 505, 506, 507, 508, 509, 511], "11": [0, 1, 2, 3, 4, 5, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 23, 24, 25, 28, 29, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41, 43, 45, 46, 47, 48, 49, 50, 51, 53, 55, 56, 58, 59, 60, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 93, 94, 95, 96, 97, 98, 99, 100, 103, 104, 105, 106, 108, 109, 110, 111, 113, 114, 115, 117, 120, 121, 123, 124, 125, 126, 127, 128, 129, 130, 131, 133, 135, 137, 138, 139, 141, 142, 144, 146, 148, 150, 152, 153, 154, 156, 157, 158, 159, 160, 162, 163, 165, 166, 167, 168, 171, 172, 174, 175, 177, 178, 179, 180, 181, 182, 183, 185, 186, 187, 188, 189, 191, 192, 193, 194, 195, 196, 197, 199, 200, 201, 202, 203, 205, 206, 207, 208, 209, 210, 211, 214, 215, 216, 218, 219, 220, 221, 223, 224, 227, 229, 230, 231, 232, 233, 234, 235, 237, 239, 240, 242, 243, 244, 246, 247, 248, 250, 251, 253, 254, 255, 256, 257, 258, 260, 261, 263, 264, 268, 269, 270, 271, 272, 273, 275, 277, 278, 279, 281, 282, 283, 284, 286, 287, 288, 291, 292, 293, 294, 295, 296, 297, 300, 301, 303, 304, 305, 306, 308, 309, 310, 311, 312, 313, 318, 319, 320, 321, 322, 323, 324, 325, 328, 329, 330, 331, 332, 334, 335, 336, 337, 339, 340, 342, 343, 344, 345, 346, 348, 349, 350, 351, 352, 354, 356, 358, 359, 360, 362, 363, 364, 365, 366, 367, 369, 371, 373, 375, 376, 377, 378, 379, 380, 382, 383, 384, 385, 386, 387, 388, 389, 390, 392, 395, 397, 398, 399, 401, 402, 403, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 418, 419, 420, 422, 423, 425, 428, 429, 430, 431, 432, 433, 434, 436, 437, 438, 440, 442, 443, 444, 450, 451, 452, 453, 454, 455, 456, 460, 462, 463, 464, 468, 469, 470, 471, 472, 474, 475, 477, 479, 480, 481, 482, 483, 485, 486, 487, 488, 489, 491, 492, 494, 496, 498, 499, 500, 501, 502, 503, 504, 506, 507, 508, 509, 510, 511], "12": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 21, 22, 23, 24, 26, 27, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 46, 47, 48, 50, 51, 52, 53, 54, 55, 60, 61, 62, 63, 64, 66, 67, 69, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 90, 91, 92, 93, 94, 95, 96, 97, 98, 103, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 116, 117, 119, 120, 121, 122, 123, 125, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 143, 144, 145, 146, 147, 148, 150, 151, 152, 153, 154, 155, 157, 160, 161, 162, 163, 165, 167, 168, 169, 170, 172, 173, 174, 175, 177, 178, 180, 181, 182, 185, 187, 188, 189, 190, 191, 192, 195, 196, 197, 198, 202, 206, 208, 209, 211, 213, 215, 216, 217, 218, 219, 220, 221, 223, 224, 225, 226, 227, 228, 230, 231, 233, 234, 235, 236, 238, 239, 240, 241, 242, 243, 244, 246, 247, 248, 249, 251, 252, 253, 254, 256, 257, 258, 259, 261, 262, 264, 265, 266, 267, 268, 269, 271, 272, 273, 274, 275, 277, 278, 279, 280, 281, 283, 284, 286, 287, 288, 289, 290, 291, 292, 294, 296, 297, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 312, 313, 314, 315, 316, 317, 318, 319, 322, 323, 324, 326, 327, 329, 331, 332, 334, 336, 338, 340, 341, 342, 343, 344, 345, 346, 347, 349, 350, 351, 352, 353, 355, 356, 357, 358, 359, 360, 363, 366, 367, 368, 369, 370, 372, 374, 376, 377, 378, 379, 380, 381, 383, 386, 388, 389, 390, 391, 392, 395, 396, 397, 399, 400, 401, 402, 404, 406, 408, 409, 410, 412, 413, 415, 416, 417, 418, 419, 421, 422, 425, 426, 429, 430, 432, 433, 434, 436, 438, 440, 442, 443, 444, 445, 446, 447, 448, 450, 452, 453, 454, 455, 456, 457, 459, 460, 462, 463, 466, 467, 468, 470, 471, 472, 475, 476, 477, 479, 481, 483, 484, 487, 488, 489, 490, 491, 493, 494, 495, 496, 497, 498, 499, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511], "13": [0, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 19, 21, 22, 24, 25, 26, 27, 29, 31, 32, 33, 34, 35, 36, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 70, 71, 72, 74, 76, 77, 78, 79, 81, 82, 83, 84, 85, 87, 90, 91, 92, 93, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 118, 121, 122, 123, 124, 127, 128, 129, 130, 131, 134, 136, 137, 138, 139, 140, 141, 142, 144, 145, 146, 147, 148, 149, 151, 154, 155, 156, 157, 159, 160, 161, 167, 168, 169, 171, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 187, 189, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 205, 206, 208, 209, 211, 212, 213, 215, 219, 220, 221, 222, 224, 225, 227, 228, 230, 232, 233, 234, 235, 237, 238, 239, 240, 241, 242, 243, 244, 246, 247, 248, 250, 251, 252, 253, 254, 255, 257, 259, 261, 263, 264, 265, 266, 267, 268, 269, 270, 272, 273, 274, 276, 277, 278, 279, 280, 281, 282, 283, 286, 287, 288, 290, 291, 292, 293, 296, 297, 301, 302, 303, 304, 305, 306, 307, 309, 310, 311, 312, 313, 316, 317, 318, 320, 321, 322, 323, 324, 325, 326, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 342, 343, 344, 346, 347, 348, 349, 350, 351, 352, 355, 356, 357, 359, 361, 362, 363, 364, 365, 366, 367, 370, 371, 372, 373, 374, 376, 377, 378, 379, 380, 381, 382, 384, 385, 387, 389, 390, 391, 392, 394, 395, 397, 398, 399, 400, 401, 402, 403, 404, 406, 409, 411, 412, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 426, 427, 428, 429, 431, 433, 434, 435, 436, 437, 438, 441, 443, 444, 445, 446, 449, 450, 451, 453, 454, 455, 457, 458, 459, 461, 462, 463, 467, 469, 470, 472, 473, 474, 475, 477, 478, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 491, 493, 495, 496, 497, 499, 500, 501, 503, 504, 506, 508, 510, 511], "14": [0, 1, 3, 4, 5, 6, 8, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 30, 32, 33, 34, 35, 36, 37, 38, 39, 41, 42, 43, 44, 45, 46, 47, 48, 50, 53, 54, 55, 56, 57, 59, 60, 63, 64, 65, 67, 68, 69, 70, 71, 73, 74, 76, 77, 78, 79, 80, 82, 83, 84, 85, 86, 88, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 102, 103, 104, 105, 106, 107, 109, 110, 113, 115, 117, 118, 119, 123, 124, 125, 126, 127, 130, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 147, 149, 150, 153, 154, 156, 157, 159, 160, 161, 162, 163, 164, 166, 167, 169, 170, 171, 172, 173, 175, 177, 181, 182, 183, 185, 186, 187, 188, 190, 191, 192, 193, 194, 195, 196, 197, 199, 200, 201, 202, 203, 204, 206, 207, 208, 209, 210, 211, 212, 213, 215, 216, 217, 218, 219, 220, 223, 226, 227, 229, 231, 233, 234, 235, 236, 238, 239, 240, 242, 243, 245, 246, 247, 249, 250, 253, 254, 255, 256, 258, 259, 260, 262, 263, 264, 267, 268, 269, 270, 272, 273, 274, 275, 276, 277, 279, 280, 281, 283, 284, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 310, 311, 315, 316, 318, 319, 320, 321, 322, 323, 324, 326, 327, 328, 329, 330, 332, 333, 334, 335, 337, 340, 341, 342, 343, 344, 345, 346, 348, 349, 350, 351, 353, 354, 355, 356, 357, 358, 361, 364, 365, 366, 367, 368, 369, 370, 372, 373, 374, 376, 377, 379, 380, 381, 383, 384, 385, 386, 387, 388, 389, 390, 391, 394, 396, 397, 398, 400, 401, 402, 403, 404, 405, 406, 408, 409, 410, 411, 412, 413, 414, 415, 417, 418, 419, 420, 421, 423, 424, 425, 426, 427, 428, 429, 431, 432, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 446, 447, 448, 450, 451, 453, 454, 455, 456, 457, 459, 461, 462, 463, 465, 467, 468, 469, 470, 471, 473, 474, 476, 478, 479, 481, 484, 485, 486, 487, 489, 491, 496, 497, 498, 500, 502, 503, 504, 505, 506, 507, 509], "15": [0, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 23, 25, 26, 27, 28, 30, 31, 32, 34, 36, 37, 39, 40, 41, 42, 43, 45, 46, 49, 50, 51, 52, 53, 56, 57, 58, 59, 61, 63, 64, 65, 66, 67, 69, 70, 71, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 84, 85, 86, 89, 90, 92, 93, 95, 96, 97, 98, 99, 100, 101, 102, 104, 105, 106, 108, 109, 110, 111, 112, 114, 115, 116, 117, 118, 119, 120, 121, 122, 124, 125, 126, 127, 128, 129, 130, 131, 132, 134, 135, 136, 137, 138, 140, 143, 144, 145, 146, 147, 149, 150, 151, 152, 154, 155, 156, 157, 158, 160, 161, 162, 163, 164, 165, 166, 168, 169, 171, 172, 175, 176, 177, 178, 179, 180, 182, 184, 185, 186, 187, 189, 190, 191, 192, 193, 194, 195, 197, 198, 199, 201, 202, 205, 206, 207, 209, 210, 211, 212, 213, 214, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 234, 236, 237, 239, 241, 242, 243, 244, 245, 246, 250, 251, 252, 255, 256, 257, 259, 260, 261, 264, 265, 266, 268, 270, 271, 273, 275, 277, 279, 280, 281, 283, 285, 287, 288, 289, 291, 292, 293, 294, 295, 296, 297, 300, 301, 302, 304, 306, 308, 309, 311, 313, 314, 315, 316, 317, 318, 319, 320, 321, 323, 324, 325, 326, 327, 329, 333, 334, 335, 336, 339, 340, 343, 344, 345, 346, 347, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 363, 364, 365, 367, 368, 369, 373, 374, 375, 377, 378, 379, 380, 381, 383, 387, 388, 389, 390, 392, 393, 394, 395, 396, 397, 399, 401, 402, 404, 406, 407, 408, 409, 410, 412, 413, 414, 415, 416, 417, 419, 420, 421, 422, 423, 426, 427, 428, 429, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 442, 443, 444, 447, 449, 450, 451, 452, 453, 454, 455, 458, 459, 460, 461, 465, 466, 467, 469, 471, 472, 473, 475, 476, 479, 480, 481, 482, 483, 484, 485, 486, 487, 489, 490, 491, 492, 494, 495, 496, 498, 499, 500, 501, 502, 503, 505, 506, 508, 509, 510, 511], "16": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 37, 38, 39, 40, 42, 43, 44, 45, 47, 49, 50, 51, 52, 53, 54, 58, 61, 62, 63, 64, 65, 67, 69, 70, 72, 73, 74, 75, 77, 79, 80, 82, 83, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 96, 97, 98, 99, 101, 104, 106, 107, 108, 109, 112, 113, 115, 116, 118, 119, 120, 121, 122, 123, 124, 127, 129, 130, 133, 135, 137, 138, 140, 142, 143, 144, 146, 147, 148, 149, 150, 151, 152, 153, 155, 156, 157, 158, 160, 162, 164, 165, 166, 167, 168, 169, 171, 172, 173, 174, 175, 176, 178, 179, 180, 181, 182, 183, 185, 187, 188, 189, 191, 192, 193, 194, 195, 196, 197, 198, 199, 201, 203, 204, 206, 207, 208, 210, 211, 212, 213, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 231, 233, 236, 237, 238, 239, 240, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 254, 255, 258, 260, 261, 262, 263, 265, 266, 267, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 281, 282, 283, 285, 286, 287, 288, 290, 292, 293, 294, 295, 296, 297, 298, 299, 300, 302, 303, 304, 306, 309, 310, 311, 312, 313, 314, 315, 316, 318, 319, 320, 322, 323, 324, 325, 326, 328, 329, 331, 333, 334, 335, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 351, 353, 355, 356, 357, 358, 360, 362, 363, 365, 368, 369, 370, 372, 374, 376, 377, 378, 380, 381, 385, 386, 387, 388, 391, 392, 393, 394, 395, 396, 397, 398, 400, 401, 402, 403, 405, 406, 407, 409, 410, 411, 413, 414, 415, 417, 418, 419, 421, 422, 423, 424, 425, 427, 428, 429, 432, 434, 437, 438, 439, 440, 441, 442, 444, 445, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 459, 460, 463, 464, 465, 466, 467, 468, 470, 471, 473, 474, 475, 476, 477, 478, 480, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 495, 500, 501, 503, 504, 505, 507, 508, 509], "17": [0, 3, 6, 7, 9, 10, 12, 13, 14, 16, 17, 18, 20, 21, 22, 23, 25, 27, 29, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 61, 62, 63, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 90, 91, 92, 93, 96, 97, 99, 101, 102, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 116, 117, 118, 120, 122, 123, 124, 125, 127, 129, 131, 132, 133, 134, 136, 137, 138, 140, 141, 143, 144, 145, 146, 147, 148, 151, 152, 154, 156, 159, 160, 163, 164, 165, 166, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 180, 182, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 196, 197, 198, 199, 201, 203, 204, 205, 207, 208, 209, 210, 211, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 231, 232, 234, 235, 236, 237, 238, 239, 240, 242, 243, 244, 245, 246, 248, 249, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 263, 264, 265, 266, 267, 268, 270, 271, 272, 273, 275, 276, 277, 278, 279, 281, 282, 283, 284, 285, 286, 288, 291, 293, 294, 295, 296, 299, 300, 301, 302, 303, 304, 305, 306, 307, 309, 310, 312, 314, 315, 319, 321, 323, 324, 325, 327, 328, 329, 331, 332, 333, 334, 335, 336, 338, 339, 340, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 368, 371, 372, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 385, 386, 387, 389, 391, 392, 394, 395, 397, 398, 401, 402, 403, 404, 405, 406, 407, 409, 410, 411, 412, 413, 414, 415, 418, 419, 420, 422, 423, 425, 426, 429, 430, 431, 432, 434, 437, 440, 442, 443, 444, 446, 448, 449, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 465, 466, 467, 468, 469, 474, 475, 477, 480, 482, 484, 485, 486, 487, 488, 490, 494, 495, 496, 497, 498, 499, 500, 501, 502, 504, 505, 506, 508, 509, 511], "18": [1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 19, 21, 22, 24, 25, 26, 28, 29, 32, 33, 34, 36, 37, 38, 39, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 53, 55, 56, 57, 58, 60, 62, 63, 65, 66, 67, 68, 69, 70, 72, 73, 75, 76, 78, 79, 80, 81, 82, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 125, 126, 127, 128, 131, 132, 133, 137, 139, 140, 141, 142, 143, 144, 145, 150, 151, 152, 153, 154, 156, 157, 160, 161, 164, 165, 166, 167, 168, 169, 170, 172, 173, 174, 176, 178, 179, 180, 182, 183, 184, 185, 186, 187, 189, 190, 193, 194, 196, 197, 198, 199, 200, 201, 203, 204, 206, 207, 208, 211, 212, 213, 214, 216, 217, 218, 219, 220, 221, 222, 223, 224, 226, 227, 229, 231, 232, 233, 234, 235, 236, 237, 238, 240, 241, 242, 244, 246, 248, 249, 250, 251, 252, 255, 256, 257, 258, 259, 260, 261, 264, 265, 266, 268, 270, 272, 274, 275, 276, 277, 278, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 302, 304, 305, 306, 307, 308, 309, 310, 311, 312, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 336, 337, 338, 340, 341, 343, 344, 345, 347, 348, 349, 350, 351, 352, 353, 354, 357, 358, 359, 361, 362, 363, 366, 368, 369, 370, 373, 374, 376, 378, 379, 380, 381, 384, 385, 386, 387, 388, 390, 391, 392, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 405, 407, 408, 409, 410, 411, 413, 414, 415, 416, 417, 418, 419, 422, 423, 425, 426, 427, 428, 430, 432, 433, 434, 435, 436, 437, 438, 441, 442, 443, 444, 447, 450, 451, 453, 456, 457, 460, 461, 463, 464, 467, 468, 470, 471, 472, 473, 475, 477, 478, 479, 480, 482, 483, 484, 485, 486, 487, 489, 492, 493, 494, 496, 497, 498, 500, 501, 503, 504, 505, 506, 507, 509, 510, 511], "19": [1, 3, 5, 6, 7, 9, 10, 11, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 40, 41, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 71, 73, 74, 75, 77, 78, 79, 80, 81, 82, 85, 87, 88, 90, 91, 92, 93, 94, 96, 97, 98, 99, 100, 101, 102, 103, 104, 106, 107, 108, 110, 111, 112, 113, 117, 118, 119, 121, 122, 123, 124, 126, 127, 129, 130, 131, 132, 133, 134, 135, 136, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 152, 153, 154, 155, 156, 157, 158, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 177, 178, 179, 181, 182, 183, 184, 185, 187, 188, 189, 190, 192, 193, 194, 195, 196, 197, 198, 199, 202, 203, 204, 205, 207, 208, 209, 210, 212, 213, 215, 216, 217, 218, 220, 221, 222, 223, 225, 226, 227, 229, 231, 233, 234, 238, 241, 242, 246, 247, 248, 249, 250, 253, 254, 255, 257, 258, 259, 260, 262, 263, 264, 265, 269, 273, 275, 277, 279, 280, 281, 282, 285, 286, 287, 288, 289, 290, 291, 292, 294, 296, 297, 298, 299, 300, 302, 304, 305, 306, 307, 308, 309, 310, 312, 313, 314, 315, 316, 317, 318, 320, 321, 322, 325, 327, 328, 329, 330, 331, 332, 334, 335, 337, 338, 339, 341, 342, 343, 347, 348, 349, 350, 353, 354, 356, 357, 359, 360, 361, 362, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 376, 377, 378, 380, 382, 383, 384, 385, 387, 388, 389, 391, 392, 395, 396, 397, 399, 401, 403, 405, 406, 407, 408, 409, 410, 411, 412, 414, 415, 416, 417, 418, 419, 420, 421, 423, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 437, 438, 440, 441, 442, 445, 446, 447, 448, 449, 450, 452, 453, 454, 456, 457, 461, 463, 464, 465, 468, 469, 470, 471, 472, 473, 479, 480, 482, 483, 484, 485, 487, 488, 489, 490, 491, 495, 497, 498, 499, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510], "20": [0, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 32, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 63, 64, 65, 66, 67, 68, 69, 71, 72, 73, 75, 76, 77, 78, 79, 80, 81, 82, 83, 85, 86, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 108, 109, 110, 112, 114, 117, 119, 121, 122, 123, 124, 125, 127, 128, 129, 130, 131, 132, 133, 134, 136, 137, 138, 139, 140, 141, 142, 143, 145, 147, 149, 151, 153, 154, 155, 156, 157, 159, 160, 161, 162, 163, 164, 165, 167, 169, 170, 172, 173, 174, 175, 176, 177, 178, 180, 181, 182, 183, 184, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 202, 203, 204, 205, 207, 208, 209, 210, 212, 213, 214, 215, 216, 217, 218, 219, 222, 223, 225, 226, 227, 228, 229, 230, 231, 235, 236, 237, 239, 240, 241, 244, 245, 247, 248, 249, 251, 252, 253, 255, 256, 258, 260, 262, 263, 265, 269, 271, 272, 273, 274, 275, 276, 277, 278, 279, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 295, 296, 297, 298, 299, 303, 305, 307, 308, 310, 311, 312, 313, 314, 315, 317, 318, 319, 320, 321, 322, 323, 325, 326, 327, 328, 329, 330, 331, 332, 334, 335, 336, 338, 339, 340, 343, 344, 345, 346, 347, 348, 349, 351, 352, 353, 355, 356, 360, 362, 363, 364, 366, 367, 369, 373, 374, 375, 379, 383, 385, 386, 387, 388, 390, 391, 394, 395, 396, 397, 399, 401, 402, 403, 404, 406, 407, 408, 409, 410, 412, 413, 414, 416, 417, 418, 419, 420, 422, 423, 426, 427, 428, 430, 431, 432, 433, 434, 435, 436, 437, 440, 441, 442, 445, 447, 448, 449, 450, 451, 453, 455, 456, 457, 458, 459, 461, 462, 463, 464, 465, 466, 467, 468, 469, 475, 477, 478, 479, 480, 482, 483, 484, 485, 486, 487, 488, 490, 493, 494, 496, 497, 498, 501, 502, 503, 504, 505, 506, 507, 508, 510, 511], "21": [0, 1, 2, 3, 4, 5, 6, 10, 11, 12, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 53, 54, 55, 56, 57, 58, 59, 61, 62, 63, 64, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 83, 84, 85, 86, 87, 88, 91, 92, 93, 94, 95, 96, 98, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 125, 126, 128, 129, 130, 131, 134, 135, 136, 138, 139, 140, 142, 144, 145, 146, 147, 148, 150, 152, 154, 155, 156, 158, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 177, 178, 180, 182, 183, 188, 189, 192, 194, 198, 199, 200, 201, 203, 204, 206, 208, 210, 211, 212, 213, 214, 215, 216, 218, 219, 220, 221, 223, 224, 225, 226, 227, 229, 231, 232, 233, 234, 236, 237, 239, 240, 242, 243, 245, 246, 247, 248, 250, 251, 252, 253, 255, 256, 257, 258, 259, 261, 264, 265, 266, 267, 268, 269, 270, 271, 275, 276, 277, 279, 280, 281, 282, 283, 284, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 298, 299, 301, 302, 304, 305, 307, 308, 309, 311, 312, 314, 316, 320, 321, 323, 324, 325, 326, 327, 329, 330, 331, 332, 333, 334, 337, 338, 339, 340, 342, 343, 345, 347, 348, 350, 351, 352, 354, 357, 358, 359, 360, 362, 363, 365, 369, 370, 371, 372, 373, 376, 377, 378, 379, 380, 382, 383, 384, 386, 387, 388, 391, 392, 393, 394, 396, 398, 400, 401, 404, 405, 407, 408, 409, 411, 412, 413, 414, 416, 417, 418, 419, 420, 421, 422, 424, 425, 426, 427, 429, 431, 432, 433, 434, 435, 436, 438, 439, 440, 441, 442, 443, 445, 446, 447, 448, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 461, 462, 463, 464, 465, 466, 467, 469, 470, 471, 472, 473, 474, 476, 477, 478, 479, 480, 481, 483, 484, 485, 488, 490, 492, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 508, 510], "22": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 31, 32, 33, 35, 36, 39, 40, 41, 42, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 62, 63, 64, 65, 66, 67, 69, 70, 73, 74, 75, 76, 77, 78, 79, 81, 82, 83, 85, 86, 91, 92, 93, 95, 96, 97, 99, 100, 101, 102, 104, 105, 106, 107, 109, 110, 111, 112, 113, 114, 115, 116, 119, 120, 121, 122, 123, 124, 125, 126, 129, 130, 133, 134, 136, 137, 138, 139, 140, 141, 142, 143, 148, 149, 150, 151, 152, 154, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 168, 169, 171, 173, 174, 175, 176, 177, 178, 179, 180, 181, 183, 186, 187, 188, 189, 190, 192, 193, 195, 196, 197, 200, 201, 203, 204, 205, 206, 208, 209, 210, 212, 215, 216, 217, 218, 219, 221, 223, 224, 225, 228, 229, 230, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 245, 246, 247, 249, 250, 252, 253, 254, 255, 256, 260, 261, 262, 264, 265, 267, 269, 270, 273, 275, 276, 279, 281, 283, 284, 285, 287, 288, 290, 291, 292, 293, 294, 295, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 311, 312, 313, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 332, 334, 335, 338, 339, 340, 343, 345, 346, 347, 348, 349, 352, 357, 358, 359, 361, 362, 364, 366, 368, 369, 370, 373, 374, 375, 377, 378, 379, 381, 383, 384, 385, 386, 387, 389, 390, 391, 392, 394, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 408, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 426, 427, 428, 429, 430, 431, 432, 434, 435, 436, 437, 440, 441, 442, 444, 445, 446, 449, 450, 452, 453, 456, 457, 458, 460, 461, 462, 463, 465, 466, 467, 468, 469, 470, 471, 472, 473, 476, 479, 481, 482, 484, 485, 487, 489, 490, 491, 492, 493, 494, 495, 496, 498, 499, 500, 501, 502, 503, 504, 506, 507, 508, 509, 511], "23": [0, 1, 3, 4, 6, 7, 9, 10, 11, 12, 13, 14, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 31, 32, 33, 34, 36, 37, 39, 41, 42, 43, 44, 45, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 62, 63, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 78, 79, 81, 82, 83, 84, 85, 91, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 107, 108, 110, 111, 112, 113, 114, 115, 117, 118, 119, 120, 121, 123, 124, 125, 126, 127, 128, 129, 131, 132, 135, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 150, 151, 152, 153, 155, 156, 158, 159, 160, 161, 162, 163, 164, 166, 167, 168, 169, 170, 172, 173, 174, 176, 177, 178, 179, 180, 181, 182, 183, 185, 186, 188, 189, 191, 193, 194, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 208, 209, 210, 211, 212, 214, 215, 217, 219, 220, 221, 222, 223, 224, 225, 226, 227, 231, 233, 234, 235, 236, 237, 238, 239, 240, 242, 244, 245, 246, 247, 249, 250, 251, 252, 255, 256, 257, 259, 260, 262, 263, 265, 266, 267, 268, 269, 271, 272, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 288, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 303, 304, 306, 308, 310, 311, 313, 315, 316, 317, 321, 323, 325, 326, 327, 328, 331, 332, 334, 335, 336, 337, 338, 339, 340, 341, 342, 344, 345, 346, 347, 349, 350, 351, 353, 354, 355, 356, 357, 358, 361, 363, 366, 367, 368, 370, 371, 372, 373, 375, 376, 377, 379, 380, 381, 382, 383, 384, 385, 386, 387, 389, 390, 391, 393, 394, 395, 397, 398, 399, 401, 402, 404, 405, 407, 409, 410, 411, 412, 413, 414, 416, 417, 419, 420, 421, 424, 425, 426, 427, 429, 430, 431, 432, 433, 434, 437, 438, 439, 440, 443, 447, 449, 450, 451, 452, 453, 455, 456, 457, 458, 459, 461, 463, 464, 465, 467, 468, 469, 470, 471, 472, 473, 476, 477, 478, 479, 480, 482, 484, 485, 486, 488, 489, 490, 492, 493, 495, 496, 497, 498, 499, 500, 501, 505, 506, 507, 508, 509, 511], "24": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 12, 13, 14, 15, 20, 23, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 37, 39, 40, 41, 43, 44, 45, 46, 48, 49, 50, 51, 52, 53, 54, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 70, 73, 74, 75, 76, 77, 78, 80, 81, 83, 85, 86, 87, 88, 89, 90, 92, 93, 94, 95, 97, 98, 99, 100, 101, 102, 104, 105, 106, 108, 109, 110, 111, 113, 115, 116, 117, 119, 121, 122, 124, 125, 126, 127, 131, 132, 133, 134, 135, 136, 138, 140, 142, 143, 144, 145, 146, 148, 151, 154, 155, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 169, 170, 171, 173, 174, 175, 176, 177, 178, 179, 181, 182, 183, 184, 186, 187, 189, 191, 193, 194, 195, 197, 198, 199, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 223, 224, 225, 227, 228, 229, 230, 231, 233, 234, 236, 237, 238, 239, 241, 242, 244, 247, 249, 251, 252, 253, 254, 255, 256, 258, 260, 261, 262, 265, 266, 267, 268, 269, 270, 272, 273, 275, 276, 277, 280, 281, 283, 287, 289, 290, 292, 293, 294, 295, 296, 297, 299, 300, 301, 303, 304, 306, 307, 308, 309, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 327, 328, 329, 330, 331, 333, 334, 335, 337, 338, 339, 340, 341, 343, 344, 345, 346, 347, 348, 349, 350, 351, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 369, 370, 371, 373, 374, 375, 376, 377, 378, 380, 381, 382, 383, 384, 386, 387, 389, 390, 391, 393, 394, 395, 397, 398, 399, 402, 403, 404, 405, 406, 408, 409, 410, 412, 413, 415, 418, 419, 420, 421, 422, 424, 425, 426, 427, 429, 430, 431, 434, 439, 442, 443, 444, 446, 447, 448, 449, 451, 452, 455, 456, 457, 458, 459, 461, 462, 463, 465, 467, 468, 469, 470, 471, 477, 478, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 493, 496, 497, 498, 499, 500, 501, 503, 505, 507, 508, 509, 510, 511], "25": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 14, 17, 18, 19, 20, 23, 24, 25, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 42, 43, 44, 45, 46, 48, 49, 50, 52, 54, 55, 56, 58, 59, 60, 62, 63, 64, 67, 68, 69, 70, 71, 72, 73, 75, 76, 77, 78, 79, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 100, 101, 102, 103, 108, 109, 111, 113, 114, 115, 116, 117, 118, 120, 122, 125, 126, 127, 128, 129, 130, 132, 133, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 148, 149, 151, 152, 153, 154, 155, 157, 158, 159, 160, 161, 162, 163, 164, 166, 167, 168, 169, 171, 172, 173, 174, 175, 176, 178, 180, 183, 184, 185, 186, 187, 189, 190, 191, 192, 193, 194, 195, 199, 200, 201, 204, 205, 206, 208, 209, 210, 211, 212, 213, 214, 215, 217, 218, 219, 220, 221, 222, 225, 228, 229, 232, 235, 236, 238, 239, 241, 242, 243, 244, 245, 246, 247, 249, 250, 251, 252, 253, 254, 255, 256, 258, 259, 260, 262, 263, 264, 266, 267, 268, 269, 270, 272, 273, 274, 279, 280, 281, 282, 283, 284, 285, 288, 291, 293, 294, 295, 296, 297, 299, 300, 301, 302, 303, 304, 305, 306, 307, 309, 310, 312, 313, 316, 317, 318, 319, 321, 323, 325, 326, 327, 328, 329, 331, 332, 333, 335, 336, 337, 339, 340, 341, 343, 346, 347, 348, 349, 352, 353, 354, 356, 357, 358, 359, 360, 361, 363, 364, 365, 366, 367, 368, 369, 371, 372, 373, 376, 377, 378, 379, 381, 382, 384, 385, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 403, 404, 405, 406, 407, 408, 410, 411, 412, 414, 415, 416, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 438, 439, 440, 442, 443, 445, 446, 447, 449, 450, 451, 453, 454, 455, 456, 457, 459, 460, 463, 464, 465, 466, 467, 470, 471, 472, 473, 474, 475, 476, 478, 481, 483, 484, 486, 487, 488, 490, 491, 493, 494, 497, 498, 499, 500, 502, 505, 506, 507, 508, 509, 510, 511], "26": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 16, 17, 18, 20, 23, 24, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 38, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 59, 60, 61, 63, 64, 65, 66, 67, 69, 70, 71, 73, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 91, 92, 94, 95, 96, 97, 98, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 113, 114, 115, 117, 119, 120, 121, 122, 123, 125, 126, 129, 130, 132, 133, 134, 136, 137, 138, 139, 140, 141, 142, 143, 146, 147, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 170, 172, 173, 174, 175, 176, 177, 178, 179, 180, 182, 184, 185, 188, 191, 193, 194, 195, 196, 197, 199, 202, 203, 205, 206, 207, 208, 209, 211, 212, 213, 214, 215, 216, 217, 218, 220, 223, 224, 227, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 242, 243, 244, 245, 246, 247, 248, 250, 251, 253, 254, 257, 258, 259, 260, 263, 264, 265, 266, 267, 268, 269, 270, 272, 273, 274, 275, 277, 278, 279, 280, 282, 284, 285, 288, 290, 291, 292, 293, 294, 296, 298, 299, 300, 301, 302, 303, 305, 306, 307, 308, 309, 310, 311, 313, 314, 315, 317, 319, 320, 321, 322, 324, 325, 326, 327, 330, 332, 333, 334, 335, 336, 337, 339, 340, 341, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 361, 362, 365, 366, 367, 369, 370, 371, 373, 374, 375, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 390, 391, 393, 394, 396, 397, 398, 399, 400, 401, 402, 403, 404, 406, 408, 409, 410, 411, 412, 413, 415, 417, 419, 420, 421, 424, 425, 427, 430, 431, 432, 433, 434, 435, 437, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 453, 454, 455, 457, 458, 459, 461, 462, 463, 464, 466, 468, 469, 471, 472, 474, 475, 476, 477, 480, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 501, 502, 503, 506, 507, 508, 509, 511], "27": [0, 1, 2, 3, 4, 5, 9, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 23, 24, 28, 29, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41, 43, 45, 46, 47, 48, 49, 50, 51, 53, 55, 56, 58, 59, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 93, 94, 95, 96, 97, 98, 99, 100, 103, 105, 108, 109, 110, 111, 113, 114, 115, 117, 120, 121, 122, 123, 124, 125, 126, 127, 129, 131, 133, 135, 137, 138, 139, 141, 142, 144, 146, 148, 150, 151, 152, 153, 154, 156, 157, 158, 159, 162, 163, 165, 167, 168, 169, 171, 172, 174, 175, 177, 178, 179, 181, 182, 183, 184, 185, 186, 187, 188, 189, 191, 192, 193, 194, 195, 196, 197, 199, 200, 201, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 218, 220, 221, 222, 223, 224, 225, 226, 227, 229, 230, 231, 232, 233, 234, 235, 237, 239, 240, 242, 243, 244, 246, 247, 248, 250, 251, 252, 253, 254, 255, 256, 257, 258, 260, 261, 262, 263, 264, 268, 269, 270, 271, 273, 274, 275, 277, 278, 279, 281, 282, 283, 286, 287, 288, 291, 292, 293, 294, 295, 296, 297, 300, 301, 303, 304, 305, 306, 308, 309, 310, 311, 312, 315, 318, 319, 320, 321, 322, 323, 324, 325, 327, 329, 330, 331, 332, 334, 335, 336, 337, 339, 340, 341, 342, 343, 344, 345, 346, 347, 349, 350, 352, 354, 357, 358, 359, 360, 361, 363, 364, 365, 367, 368, 369, 371, 373, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 392, 395, 397, 398, 399, 401, 403, 404, 406, 407, 408, 409, 410, 412, 413, 414, 415, 416, 418, 419, 420, 421, 422, 423, 424, 425, 428, 429, 430, 432, 433, 434, 436, 437, 438, 439, 440, 442, 443, 444, 448, 451, 452, 453, 454, 455, 456, 460, 462, 463, 464, 468, 469, 470, 471, 472, 474, 475, 477, 478, 479, 480, 482, 483, 484, 486, 489, 491, 492, 494, 496, 498, 499, 500, 501, 502, 503, 504, 506, 507, 508, 509, 510, 511], "28": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 21, 22, 23, 24, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 47, 48, 50, 51, 52, 53, 55, 60, 61, 62, 63, 64, 65, 66, 67, 69, 70, 71, 73, 74, 75, 76, 77, 78, 80, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 103, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 116, 117, 119, 120, 121, 122, 123, 125, 127, 130, 132, 133, 134, 135, 136, 137, 139, 140, 141, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 158, 160, 161, 162, 163, 165, 167, 168, 169, 170, 172, 173, 174, 175, 177, 178, 180, 181, 182, 185, 187, 188, 189, 190, 191, 192, 196, 197, 198, 200, 201, 202, 205, 206, 208, 209, 211, 215, 216, 217, 218, 219, 220, 221, 223, 224, 226, 227, 228, 230, 231, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 246, 247, 248, 249, 251, 252, 253, 254, 256, 257, 258, 259, 261, 262, 264, 265, 266, 267, 268, 269, 271, 272, 273, 274, 275, 277, 278, 280, 281, 282, 284, 285, 286, 287, 288, 289, 290, 291, 292, 294, 295, 296, 297, 299, 301, 302, 303, 304, 305, 306, 307, 312, 313, 314, 315, 316, 317, 318, 319, 322, 323, 324, 325, 326, 327, 331, 332, 334, 335, 336, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 349, 350, 351, 352, 353, 355, 356, 357, 360, 363, 366, 367, 369, 370, 372, 374, 377, 378, 379, 380, 381, 386, 387, 388, 389, 390, 391, 392, 395, 396, 397, 399, 400, 401, 402, 404, 406, 407, 408, 409, 410, 412, 413, 414, 415, 416, 417, 418, 419, 421, 422, 423, 425, 426, 429, 430, 432, 433, 434, 436, 438, 440, 442, 443, 444, 445, 446, 447, 448, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 465, 466, 467, 468, 469, 470, 471, 472, 473, 475, 476, 477, 479, 480, 481, 483, 484, 488, 489, 491, 492, 493, 494, 495, 497, 498, 499, 501, 502, 503, 505, 506, 507, 508, 509, 510, 511], "29": [0, 2, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 29, 31, 32, 33, 34, 37, 38, 40, 41, 42, 44, 45, 46, 47, 48, 50, 51, 52, 53, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 69, 70, 71, 72, 74, 76, 77, 78, 79, 81, 82, 83, 84, 85, 87, 90, 91, 92, 93, 95, 96, 97, 98, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 115, 116, 118, 120, 121, 122, 123, 124, 125, 127, 128, 129, 130, 131, 132, 134, 135, 136, 137, 138, 139, 140, 141, 142, 144, 145, 146, 147, 148, 149, 151, 152, 154, 155, 156, 157, 158, 159, 160, 161, 166, 167, 168, 169, 171, 173, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 190, 193, 194, 195, 196, 197, 198, 199, 201, 203, 205, 208, 209, 211, 212, 213, 215, 216, 220, 221, 222, 224, 225, 226, 228, 230, 231, 232, 233, 234, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 257, 259, 260, 261, 263, 264, 265, 266, 268, 269, 270, 272, 273, 274, 275, 276, 277, 278, 280, 281, 282, 283, 286, 287, 288, 290, 291, 292, 293, 296, 297, 301, 302, 303, 304, 305, 306, 307, 309, 310, 312, 313, 314, 315, 316, 318, 320, 321, 322, 323, 324, 325, 328, 329, 330, 331, 332, 334, 335, 336, 337, 339, 342, 343, 344, 346, 347, 348, 350, 351, 352, 355, 356, 359, 361, 362, 363, 364, 365, 367, 368, 370, 371, 372, 373, 375, 376, 377, 378, 379, 380, 381, 382, 384, 385, 386, 387, 389, 390, 391, 392, 394, 395, 397, 398, 399, 401, 402, 403, 404, 406, 409, 411, 412, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 426, 427, 428, 429, 433, 434, 435, 436, 438, 440, 441, 443, 444, 445, 446, 447, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 461, 462, 463, 467, 469, 470, 472, 473, 474, 475, 476, 477, 479, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 493, 495, 496, 497, 499, 500, 501, 503, 504, 506, 510, 511], "30": [0, 1, 4, 5, 6, 8, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41, 42, 43, 44, 45, 46, 47, 48, 50, 52, 53, 54, 55, 56, 57, 59, 60, 62, 64, 65, 67, 68, 69, 70, 71, 73, 74, 76, 77, 78, 80, 82, 83, 84, 85, 86, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 102, 103, 104, 105, 106, 107, 109, 110, 112, 113, 114, 115, 117, 118, 119, 124, 125, 127, 128, 129, 130, 132, 133, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 149, 150, 151, 153, 154, 156, 157, 159, 160, 161, 162, 164, 166, 167, 169, 170, 171, 172, 173, 175, 177, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 199, 200, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 216, 217, 218, 219, 220, 226, 227, 229, 230, 231, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 249, 250, 253, 254, 255, 256, 258, 259, 260, 262, 263, 264, 267, 268, 269, 270, 272, 273, 274, 276, 277, 278, 279, 280, 281, 283, 284, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 311, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 337, 340, 341, 342, 343, 345, 346, 348, 349, 350, 351, 354, 356, 357, 358, 361, 363, 364, 366, 367, 368, 369, 370, 374, 376, 377, 379, 380, 381, 383, 384, 385, 386, 387, 388, 389, 390, 391, 394, 395, 396, 397, 398, 400, 401, 402, 403, 404, 405, 406, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 419, 420, 421, 423, 424, 425, 426, 427, 428, 432, 434, 435, 436, 437, 438, 439, 440, 441, 443, 444, 446, 447, 448, 450, 451, 453, 454, 455, 456, 457, 459, 460, 461, 462, 463, 467, 468, 469, 470, 471, 473, 474, 476, 478, 479, 481, 484, 486, 487, 489, 497, 498, 499, 500, 502, 503, 504, 505, 506, 507, 508, 509, 511], "31": [0, 2, 3, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 30, 31, 32, 34, 36, 37, 39, 40, 41, 42, 43, 44, 45, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 63, 64, 65, 66, 67, 68, 70, 73, 74, 76, 77, 78, 79, 80, 81, 82, 84, 85, 86, 88, 89, 90, 93, 95, 96, 97, 98, 99, 100, 102, 105, 106, 108, 109, 110, 111, 112, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 126, 127, 129, 130, 131, 132, 134, 135, 136, 137, 138, 140, 143, 144, 145, 146, 147, 149, 150, 152, 154, 155, 156, 157, 158, 159, 160, 161, 162, 164, 165, 166, 168, 169, 170, 171, 172, 173, 175, 176, 177, 178, 179, 180, 182, 184, 185, 186, 187, 190, 191, 192, 193, 194, 195, 197, 198, 199, 202, 205, 206, 207, 209, 210, 211, 212, 213, 214, 215, 217, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 234, 236, 237, 239, 241, 242, 243, 244, 245, 246, 247, 250, 251, 252, 255, 256, 257, 259, 260, 261, 263, 264, 265, 266, 268, 269, 270, 271, 272, 273, 275, 279, 280, 281, 282, 283, 284, 285, 287, 288, 289, 291, 292, 293, 294, 295, 296, 297, 300, 301, 302, 303, 304, 306, 308, 309, 311, 313, 314, 315, 316, 317, 318, 319, 320, 321, 323, 324, 325, 326, 327, 328, 329, 333, 334, 335, 336, 339, 340, 343, 345, 346, 347, 348, 349, 350, 351, 352, 354, 355, 356, 357, 358, 359, 360, 361, 363, 364, 365, 367, 368, 369, 371, 373, 374, 375, 377, 378, 379, 380, 381, 383, 387, 388, 389, 390, 391, 392, 393, 394, 396, 397, 399, 401, 402, 405, 407, 408, 409, 410, 412, 413, 414, 415, 416, 417, 419, 420, 421, 422, 423, 424, 426, 427, 428, 429, 431, 433, 434, 435, 436, 437, 438, 439, 440, 442, 443, 444, 446, 449, 450, 451, 452, 453, 454, 455, 458, 459, 461, 465, 466, 467, 469, 470, 471, 472, 473, 475, 476, 479, 480, 481, 482, 483, 485, 486, 487, 489, 490, 491, 492, 494, 495, 496, 498, 500, 501, 502, 505, 506, 508, 509, 510], "32": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 12, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 35, 36, 37, 39, 40, 42, 43, 44, 45, 47, 49, 50, 51, 52, 53, 54, 58, 61, 62, 63, 64, 65, 67, 69, 70, 71, 72, 73, 74, 75, 79, 80, 82, 83, 86, 87, 88, 89, 90, 91, 92, 94, 96, 97, 98, 99, 101, 104, 106, 107, 108, 109, 110, 111, 112, 115, 116, 118, 119, 120, 121, 122, 123, 124, 127, 129, 130, 133, 135, 136, 137, 138, 140, 142, 143, 144, 146, 147, 148, 150, 151, 152, 153, 155, 157, 158, 159, 160, 162, 163, 164, 165, 166, 167, 168, 169, 171, 172, 173, 174, 175, 176, 178, 179, 180, 181, 182, 183, 185, 187, 189, 190, 191, 192, 193, 194, 195, 196, 197, 199, 201, 203, 204, 206, 207, 208, 210, 211, 212, 213, 216, 218, 219, 220, 221, 222, 224, 225, 226, 227, 228, 229, 230, 231, 233, 234, 235, 236, 237, 238, 239, 240, 241, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 255, 256, 260, 261, 262, 263, 265, 266, 267, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 281, 282, 283, 285, 286, 287, 288, 290, 293, 294, 295, 296, 298, 299, 300, 302, 303, 304, 309, 310, 311, 312, 313, 314, 315, 316, 317, 319, 320, 322, 323, 324, 325, 326, 328, 329, 331, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 351, 353, 354, 355, 356, 357, 358, 359, 360, 362, 363, 368, 369, 370, 372, 374, 375, 376, 377, 378, 379, 380, 381, 385, 386, 387, 388, 391, 392, 393, 394, 395, 396, 397, 398, 400, 401, 402, 403, 405, 406, 407, 409, 410, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 427, 428, 429, 432, 434, 437, 438, 439, 440, 441, 442, 444, 445, 446, 448, 450, 451, 452, 453, 454, 455, 456, 457, 459, 460, 463, 464, 465, 466, 467, 468, 470, 471, 473, 475, 476, 477, 478, 479, 480, 482, 483, 484, 486, 487, 488, 489, 490, 491, 492, 495, 500, 501, 503, 504, 505, 506, 507, 508, 509, 511], "33": [0, 3, 6, 7, 9, 10, 12, 14, 16, 17, 18, 20, 21, 22, 23, 25, 27, 28, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 61, 62, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 96, 97, 99, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 136, 137, 138, 139, 140, 141, 144, 145, 146, 147, 148, 151, 152, 154, 156, 158, 159, 160, 163, 164, 165, 166, 168, 170, 171, 172, 173, 174, 175, 176, 177, 178, 180, 182, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 196, 198, 199, 201, 202, 203, 204, 205, 207, 208, 209, 210, 211, 214, 215, 216, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 231, 232, 234, 235, 236, 237, 238, 239, 240, 243, 244, 245, 246, 249, 252, 253, 254, 255, 256, 257, 259, 260, 261, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 275, 276, 277, 278, 279, 281, 282, 283, 284, 285, 286, 288, 291, 293, 294, 296, 299, 300, 301, 302, 303, 304, 305, 306, 309, 311, 312, 314, 315, 317, 319, 321, 323, 324, 325, 327, 328, 329, 331, 332, 333, 334, 335, 336, 338, 339, 342, 343, 344, 345, 346, 347, 349, 350, 351, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 368, 371, 372, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 385, 386, 387, 389, 391, 392, 393, 394, 395, 398, 401, 402, 403, 404, 406, 407, 409, 410, 411, 412, 414, 415, 418, 419, 420, 422, 423, 425, 426, 427, 430, 431, 432, 434, 437, 440, 441, 442, 443, 444, 446, 449, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 465, 466, 467, 468, 469, 471, 474, 475, 477, 480, 482, 483, 484, 485, 486, 487, 488, 490, 494, 495, 496, 497, 498, 499, 500, 501, 503, 504, 505, 506, 508, 509, 511], "34": [1, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 17, 18, 19, 20, 22, 24, 25, 26, 28, 29, 32, 33, 34, 36, 37, 38, 39, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 53, 55, 56, 57, 58, 60, 62, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 76, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 113, 115, 116, 117, 118, 119, 120, 121, 122, 123, 125, 126, 127, 128, 130, 131, 132, 133, 137, 139, 140, 141, 142, 143, 144, 145, 146, 150, 151, 152, 153, 154, 156, 157, 160, 161, 164, 165, 166, 167, 168, 169, 170, 172, 173, 174, 176, 178, 179, 180, 183, 184, 185, 186, 187, 188, 189, 190, 192, 193, 194, 196, 197, 198, 199, 200, 201, 203, 204, 206, 207, 208, 210, 211, 212, 213, 214, 216, 217, 218, 219, 220, 221, 222, 223, 224, 226, 227, 229, 231, 232, 233, 234, 235, 236, 238, 239, 240, 241, 244, 246, 248, 249, 250, 251, 253, 255, 256, 258, 259, 260, 261, 264, 265, 266, 267, 268, 270, 272, 275, 276, 277, 278, 280, 281, 282, 283, 285, 286, 287, 288, 289, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 304, 305, 306, 307, 308, 309, 310, 311, 312, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 336, 337, 338, 340, 341, 343, 344, 345, 347, 349, 350, 351, 353, 357, 358, 359, 361, 362, 363, 366, 368, 369, 370, 373, 374, 376, 378, 379, 380, 381, 384, 385, 386, 388, 390, 391, 392, 394, 395, 397, 399, 400, 401, 402, 403, 405, 407, 408, 409, 410, 411, 413, 414, 415, 416, 417, 418, 419, 422, 423, 425, 426, 428, 430, 433, 434, 435, 436, 437, 438, 441, 442, 443, 444, 447, 449, 451, 453, 456, 457, 459, 460, 461, 462, 463, 464, 467, 468, 470, 471, 472, 473, 475, 477, 478, 479, 480, 482, 483, 484, 485, 486, 487, 488, 489, 492, 493, 494, 496, 497, 498, 500, 501, 503, 504, 505, 506, 507, 509, 510, 511], "35": [1, 3, 6, 7, 9, 10, 11, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 40, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 56, 57, 58, 59, 60, 61, 62, 63, 65, 66, 67, 68, 69, 71, 73, 77, 78, 79, 80, 81, 82, 85, 87, 88, 89, 90, 91, 92, 93, 94, 96, 97, 98, 99, 100, 101, 102, 103, 104, 106, 107, 108, 110, 111, 112, 113, 117, 118, 119, 121, 122, 123, 124, 126, 127, 129, 130, 132, 133, 134, 135, 136, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 152, 154, 155, 156, 157, 160, 161, 162, 163, 164, 165, 166, 167, 169, 171, 172, 173, 174, 175, 178, 179, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 202, 203, 205, 207, 208, 209, 210, 212, 213, 215, 216, 217, 218, 220, 221, 222, 223, 225, 226, 227, 228, 229, 231, 232, 233, 234, 236, 237, 238, 241, 242, 245, 246, 247, 248, 249, 250, 253, 254, 255, 257, 258, 259, 260, 261, 262, 263, 264, 265, 269, 273, 274, 275, 277, 279, 280, 282, 285, 286, 287, 288, 289, 290, 291, 292, 294, 296, 297, 298, 299, 300, 302, 303, 305, 306, 307, 308, 309, 310, 312, 313, 314, 315, 316, 317, 318, 320, 321, 322, 324, 325, 327, 328, 329, 330, 331, 332, 334, 335, 338, 339, 341, 342, 343, 345, 347, 348, 349, 350, 354, 356, 357, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 376, 377, 378, 380, 382, 383, 384, 385, 386, 387, 388, 389, 391, 392, 395, 396, 397, 399, 401, 403, 405, 406, 407, 408, 409, 410, 411, 412, 414, 415, 416, 417, 418, 419, 420, 421, 423, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 438, 440, 442, 443, 445, 446, 447, 448, 449, 450, 452, 453, 454, 455, 456, 457, 461, 462, 463, 464, 465, 468, 469, 470, 471, 472, 473, 474, 479, 480, 482, 483, 484, 485, 487, 488, 489, 494, 495, 497, 498, 499, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511], "36": [0, 2, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 55, 56, 57, 58, 59, 60, 61, 63, 64, 65, 66, 68, 69, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 85, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 117, 119, 121, 122, 123, 124, 125, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 149, 151, 152, 153, 154, 155, 156, 157, 159, 160, 161, 162, 164, 165, 167, 169, 170, 172, 173, 174, 175, 176, 177, 178, 180, 182, 183, 184, 189, 190, 191, 192, 193, 194, 195, 197, 198, 199, 200, 202, 203, 204, 205, 207, 208, 209, 210, 212, 213, 214, 215, 216, 217, 218, 219, 222, 223, 225, 226, 227, 228, 230, 231, 235, 236, 239, 240, 241, 243, 244, 245, 247, 248, 249, 251, 252, 253, 255, 256, 258, 260, 262, 263, 265, 269, 271, 272, 273, 274, 275, 276, 277, 278, 279, 282, 283, 284, 285, 286, 288, 289, 290, 291, 292, 295, 296, 297, 298, 299, 300, 302, 303, 305, 306, 307, 308, 310, 311, 312, 313, 315, 317, 318, 319, 320, 321, 322, 323, 325, 326, 327, 328, 329, 331, 332, 334, 335, 336, 338, 339, 340, 343, 344, 345, 346, 347, 348, 349, 353, 355, 356, 358, 360, 362, 363, 366, 367, 368, 369, 373, 374, 375, 378, 379, 383, 385, 386, 388, 390, 391, 394, 395, 396, 397, 399, 401, 402, 403, 404, 406, 407, 408, 409, 412, 413, 414, 416, 417, 418, 419, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 436, 437, 440, 441, 442, 445, 447, 448, 449, 450, 451, 453, 454, 455, 456, 457, 458, 459, 461, 462, 463, 464, 465, 466, 467, 468, 469, 475, 477, 478, 479, 482, 483, 484, 485, 486, 487, 488, 490, 493, 494, 496, 497, 498, 499, 501, 502, 503, 504, 505, 506, 507, 508, 510, 511], "37": [0, 1, 2, 3, 4, 5, 6, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 48, 49, 53, 54, 55, 56, 57, 58, 59, 61, 62, 63, 64, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 81, 83, 84, 85, 86, 87, 88, 90, 91, 92, 93, 94, 95, 96, 97, 98, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 125, 128, 129, 130, 131, 134, 135, 136, 138, 139, 140, 142, 144, 145, 146, 147, 148, 150, 152, 154, 155, 156, 157, 158, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 172, 173, 174, 175, 176, 177, 178, 180, 182, 183, 188, 189, 192, 194, 198, 199, 200, 201, 203, 204, 206, 208, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 223, 225, 226, 227, 229, 231, 232, 233, 234, 236, 237, 238, 239, 240, 241, 242, 243, 245, 246, 247, 248, 250, 251, 252, 253, 255, 257, 261, 264, 265, 266, 267, 268, 270, 271, 275, 276, 277, 279, 280, 281, 282, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 298, 299, 301, 302, 303, 304, 305, 306, 307, 308, 309, 311, 312, 313, 314, 316, 320, 321, 323, 324, 325, 326, 327, 329, 330, 331, 332, 333, 334, 337, 338, 339, 340, 342, 343, 344, 345, 347, 348, 350, 351, 352, 354, 357, 358, 359, 360, 362, 365, 369, 370, 371, 372, 373, 377, 378, 379, 380, 381, 382, 383, 384, 386, 387, 388, 389, 391, 392, 393, 394, 396, 398, 400, 401, 404, 405, 407, 408, 409, 411, 412, 413, 414, 416, 417, 418, 419, 420, 421, 422, 424, 425, 426, 427, 429, 431, 432, 433, 434, 435, 436, 438, 439, 440, 441, 442, 445, 446, 447, 448, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 461, 462, 463, 464, 465, 466, 468, 469, 470, 471, 472, 473, 474, 476, 477, 478, 479, 480, 481, 483, 484, 485, 490, 492, 494, 496, 498, 499, 500, 501, 502, 503, 504, 506, 508, 510], "38": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30, 31, 32, 33, 35, 36, 39, 40, 41, 42, 43, 44, 45, 46, 48, 49, 51, 52, 53, 54, 56, 57, 58, 59, 60, 62, 63, 64, 65, 66, 67, 69, 73, 74, 75, 76, 77, 78, 79, 81, 82, 83, 85, 86, 90, 91, 92, 93, 95, 96, 97, 99, 100, 101, 102, 104, 105, 106, 107, 109, 110, 111, 112, 113, 114, 115, 116, 119, 120, 121, 122, 123, 124, 125, 126, 129, 130, 133, 134, 136, 137, 138, 139, 140, 141, 142, 143, 147, 148, 149, 150, 151, 152, 154, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 168, 169, 171, 173, 174, 175, 176, 177, 178, 179, 180, 181, 186, 187, 188, 189, 190, 192, 193, 195, 196, 197, 200, 201, 203, 204, 205, 206, 208, 209, 210, 211, 212, 214, 215, 216, 217, 218, 221, 223, 224, 225, 226, 228, 229, 230, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 245, 246, 247, 249, 250, 252, 253, 254, 255, 256, 260, 261, 262, 264, 265, 266, 267, 269, 270, 271, 273, 275, 276, 279, 281, 283, 284, 285, 287, 288, 290, 291, 292, 293, 294, 295, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 311, 312, 313, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 332, 334, 335, 338, 339, 340, 342, 343, 345, 346, 347, 348, 349, 352, 354, 357, 358, 359, 361, 362, 363, 364, 366, 368, 369, 370, 372, 373, 374, 375, 377, 379, 381, 383, 384, 385, 386, 387, 389, 390, 391, 392, 394, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 426, 427, 428, 429, 430, 431, 432, 434, 435, 436, 437, 440, 441, 442, 444, 446, 449, 450, 452, 453, 456, 457, 458, 460, 461, 462, 463, 465, 466, 467, 468, 469, 470, 471, 472, 473, 476, 479, 481, 482, 485, 489, 490, 491, 493, 494, 495, 496, 499, 500, 501, 502, 503, 504, 506, 507, 508, 509, 511], "39": [0, 1, 2, 3, 4, 5, 6, 7, 9, 11, 12, 13, 16, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 31, 32, 33, 34, 36, 39, 40, 41, 42, 44, 45, 46, 50, 51, 52, 53, 55, 56, 57, 58, 59, 60, 62, 63, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 81, 82, 83, 84, 85, 87, 89, 91, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 107, 108, 110, 111, 112, 114, 115, 117, 118, 119, 120, 121, 123, 124, 125, 126, 128, 131, 132, 134, 135, 137, 139, 140, 141, 142, 143, 144, 145, 146, 147, 149, 151, 152, 153, 155, 156, 158, 159, 160, 161, 162, 163, 164, 166, 167, 168, 169, 170, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 185, 186, 188, 190, 191, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 208, 209, 210, 211, 212, 214, 215, 217, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 231, 234, 235, 236, 237, 238, 239, 242, 244, 245, 246, 247, 249, 250, 251, 254, 255, 256, 257, 258, 260, 262, 263, 264, 265, 267, 268, 269, 271, 272, 274, 275, 276, 277, 278, 279, 280, 281, 283, 284, 285, 286, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 302, 303, 304, 306, 308, 310, 311, 313, 315, 316, 318, 321, 323, 326, 327, 328, 331, 332, 334, 335, 336, 337, 338, 339, 340, 341, 342, 344, 345, 346, 347, 349, 350, 351, 353, 354, 355, 356, 357, 358, 361, 363, 366, 367, 368, 369, 370, 371, 372, 373, 375, 376, 377, 380, 381, 382, 383, 384, 385, 386, 387, 389, 390, 391, 393, 394, 395, 397, 398, 399, 400, 401, 402, 404, 405, 406, 407, 409, 410, 411, 412, 413, 414, 416, 417, 419, 421, 425, 426, 427, 429, 430, 431, 432, 433, 434, 436, 437, 438, 439, 440, 441, 443, 444, 447, 449, 450, 452, 453, 454, 455, 456, 457, 458, 459, 461, 463, 464, 467, 468, 469, 470, 471, 472, 473, 477, 478, 479, 480, 481, 482, 484, 485, 486, 488, 489, 490, 492, 493, 495, 496, 497, 498, 499, 500, 501, 505, 506, 508, 509, 510, 511], "40": [1, 2, 3, 4, 6, 9, 13, 14, 16, 18, 19, 20, 21, 23, 25, 26, 27, 28, 29, 30, 31, 32, 34, 35, 36, 37, 39, 40, 42, 43, 44, 45, 48, 49, 50, 51, 52, 53, 54, 55, 56, 58, 59, 60, 61, 62, 63, 65, 67, 69, 70, 73, 74, 75, 76, 77, 79, 80, 81, 82, 83, 84, 85, 87, 88, 89, 90, 91, 92, 93, 95, 96, 98, 100, 103, 105, 106, 108, 111, 113, 114, 116, 117, 118, 119, 120, 122, 123, 126, 127, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 143, 144, 145, 146, 147, 148, 151, 153, 155, 157, 158, 159, 160, 161, 162, 164, 165, 166, 168, 169, 170, 171, 172, 173, 174, 176, 177, 178, 179, 180, 181, 182, 183, 185, 186, 187, 188, 189, 190, 191, 192, 194, 195, 197, 198, 199, 200, 201, 202, 203, 205, 206, 208, 209, 211, 212, 213, 214, 215, 217, 219, 221, 222, 223, 226, 227, 228, 229, 230, 231, 234, 235, 238, 239, 240, 244, 245, 247, 248, 249, 250, 251, 252, 254, 255, 256, 257, 258, 259, 261, 262, 264, 265, 266, 267, 268, 270, 271, 272, 273, 274, 275, 278, 279, 280, 281, 283, 285, 286, 287, 288, 289, 290, 291, 292, 294, 295, 296, 297, 298, 299, 300, 302, 303, 304, 306, 308, 309, 311, 313, 314, 315, 316, 317, 318, 320, 323, 324, 325, 327, 328, 329, 330, 332, 333, 334, 335, 336, 338, 340, 341, 342, 344, 346, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 359, 362, 363, 364, 365, 366, 367, 373, 376, 377, 378, 379, 380, 382, 383, 384, 385, 386, 388, 389, 390, 391, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 404, 406, 407, 410, 411, 414, 415, 416, 417, 418, 419, 420, 422, 423, 425, 426, 427, 428, 429, 430, 431, 433, 434, 435, 436, 437, 438, 439, 441, 442, 443, 445, 447, 448, 449, 450, 451, 452, 453, 456, 457, 458, 459, 460, 461, 462, 463, 464, 466, 467, 470, 471, 472, 473, 474, 475, 477, 478, 479, 480, 481, 482, 483, 485, 486, 487, 488, 489, 490, 491, 493, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 507, 508, 509, 511], "41": [0, 1, 2, 3, 5, 10, 12, 15, 16, 17, 18, 19, 20, 21, 22, 24, 25, 27, 29, 32, 33, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 58, 59, 61, 62, 64, 65, 67, 68, 70, 71, 73, 74, 77, 78, 79, 80, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 93, 94, 95, 96, 97, 98, 99, 100, 102, 103, 107, 109, 111, 112, 115, 117, 118, 119, 124, 125, 128, 129, 132, 133, 134, 135, 136, 137, 138, 140, 143, 144, 145, 146, 147, 149, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 162, 163, 164, 166, 167, 168, 169, 170, 171, 175, 176, 178, 179, 180, 181, 182, 183, 186, 188, 189, 190, 191, 193, 194, 196, 197, 198, 199, 200, 201, 203, 204, 205, 206, 207, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 226, 227, 228, 229, 230, 231, 233, 235, 237, 238, 240, 241, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 272, 273, 274, 278, 279, 281, 282, 283, 284, 287, 289, 290, 291, 292, 293, 294, 295, 296, 297, 299, 300, 301, 302, 303, 304, 305, 306, 308, 310, 311, 313, 314, 316, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 338, 339, 340, 342, 343, 344, 346, 348, 349, 350, 351, 352, 353, 354, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 372, 374, 375, 377, 378, 380, 381, 382, 383, 384, 385, 386, 387, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 407, 408, 409, 410, 411, 414, 415, 417, 418, 419, 420, 422, 423, 424, 425, 426, 427, 429, 430, 431, 432, 433, 434, 435, 437, 439, 440, 441, 443, 444, 445, 446, 448, 449, 450, 451, 454, 457, 458, 459, 462, 463, 464, 465, 467, 468, 470, 471, 472, 474, 475, 476, 478, 479, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 499, 500, 501, 504, 505, 507, 508, 509, 511], "42": [3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30, 31, 33, 34, 35, 36, 37, 38, 39, 40, 42, 43, 45, 46, 47, 49, 50, 52, 53, 55, 56, 57, 58, 59, 61, 62, 67, 68, 69, 70, 71, 73, 75, 76, 77, 78, 79, 80, 83, 84, 85, 86, 87, 88, 90, 91, 92, 93, 95, 96, 97, 98, 99, 102, 103, 104, 105, 106, 110, 111, 113, 114, 116, 119, 120, 123, 124, 125, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 146, 148, 150, 151, 152, 154, 157, 158, 159, 162, 163, 164, 165, 168, 169, 170, 171, 172, 173, 174, 175, 177, 178, 180, 181, 183, 184, 185, 186, 187, 189, 190, 192, 194, 195, 197, 198, 199, 200, 201, 202, 203, 204, 205, 207, 208, 210, 211, 212, 214, 215, 216, 217, 218, 220, 222, 225, 226, 228, 231, 232, 233, 234, 235, 236, 237, 238, 241, 242, 243, 244, 245, 246, 248, 249, 250, 251, 252, 255, 256, 257, 258, 259, 260, 261, 263, 264, 265, 266, 267, 268, 271, 272, 273, 274, 276, 277, 279, 283, 284, 285, 286, 287, 289, 290, 291, 293, 294, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 307, 309, 310, 312, 313, 316, 317, 318, 319, 320, 321, 322, 323, 325, 326, 327, 328, 329, 331, 332, 334, 335, 338, 339, 340, 341, 342, 344, 345, 346, 347, 348, 349, 351, 353, 354, 355, 356, 357, 358, 359, 360, 362, 363, 364, 365, 366, 367, 368, 369, 373, 374, 376, 378, 379, 380, 381, 382, 384, 386, 387, 388, 389, 390, 391, 393, 394, 395, 396, 397, 401, 402, 403, 404, 406, 410, 411, 412, 413, 415, 416, 417, 418, 419, 420, 422, 423, 424, 425, 426, 428, 429, 430, 431, 432, 433, 434, 435, 436, 438, 439, 441, 442, 444, 446, 447, 448, 449, 453, 455, 456, 457, 458, 462, 463, 464, 465, 466, 469, 470, 471, 472, 473, 474, 475, 476, 478, 479, 480, 481, 482, 483, 484, 487, 488, 489, 490, 491, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 504, 505, 506, 508, 509, 510, 511], "43": [1, 2, 4, 5, 6, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 24, 25, 26, 28, 29, 30, 31, 32, 34, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 53, 54, 55, 56, 57, 60, 64, 65, 66, 67, 70, 71, 73, 75, 77, 79, 81, 83, 84, 85, 86, 87, 89, 91, 92, 93, 94, 95, 96, 98, 99, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 135, 136, 137, 139, 141, 142, 144, 145, 146, 148, 149, 150, 154, 156, 157, 159, 161, 162, 163, 164, 166, 167, 168, 169, 171, 172, 173, 174, 175, 177, 178, 179, 180, 181, 182, 183, 185, 186, 189, 190, 191, 193, 194, 196, 198, 199, 202, 203, 204, 205, 206, 207, 208, 209, 210, 212, 213, 214, 215, 216, 217, 219, 220, 221, 223, 224, 225, 226, 229, 230, 231, 232, 233, 234, 237, 238, 239, 241, 242, 243, 244, 245, 246, 247, 248, 249, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 265, 266, 267, 269, 270, 271, 273, 274, 275, 276, 277, 278, 280, 282, 283, 284, 285, 286, 287, 290, 292, 293, 294, 295, 296, 297, 300, 301, 302, 303, 305, 306, 308, 310, 311, 312, 315, 316, 319, 320, 322, 324, 326, 329, 330, 331, 332, 333, 335, 336, 337, 339, 340, 341, 342, 345, 346, 349, 351, 354, 355, 357, 358, 359, 360, 361, 362, 363, 365, 366, 367, 369, 370, 371, 372, 374, 376, 377, 378, 380, 381, 382, 383, 384, 385, 386, 387, 388, 391, 393, 394, 395, 397, 398, 399, 400, 404, 405, 406, 407, 408, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 424, 425, 426, 427, 428, 429, 430, 431, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 451, 452, 453, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 469, 471, 472, 474, 475, 476, 477, 478, 481, 482, 483, 484, 485, 486, 487, 489, 490, 491, 492, 494, 495, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511], "44": [1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 17, 19, 23, 24, 25, 26, 27, 28, 30, 31, 33, 34, 35, 37, 38, 39, 40, 43, 44, 45, 46, 47, 49, 50, 51, 52, 53, 54, 55, 56, 57, 59, 60, 61, 64, 65, 68, 70, 71, 72, 73, 74, 75, 76, 78, 79, 81, 82, 84, 87, 89, 90, 91, 92, 93, 94, 95, 96, 97, 99, 100, 103, 104, 105, 106, 108, 109, 110, 113, 116, 117, 119, 120, 121, 122, 123, 124, 126, 127, 129, 132, 133, 134, 135, 136, 140, 141, 142, 144, 145, 146, 147, 148, 150, 152, 153, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 172, 173, 174, 175, 176, 177, 178, 180, 181, 182, 183, 184, 186, 187, 189, 190, 191, 194, 195, 196, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 212, 213, 214, 216, 217, 218, 220, 221, 222, 223, 224, 225, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 240, 242, 243, 244, 245, 247, 248, 249, 250, 251, 252, 253, 255, 256, 258, 259, 260, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 296, 297, 298, 299, 300, 301, 302, 303, 305, 307, 308, 309, 311, 312, 313, 315, 317, 319, 320, 322, 324, 326, 327, 329, 330, 331, 332, 333, 335, 336, 337, 338, 339, 340, 342, 344, 346, 347, 348, 349, 350, 351, 352, 353, 354, 356, 358, 359, 361, 362, 364, 366, 368, 370, 372, 376, 378, 379, 382, 383, 384, 385, 386, 387, 389, 392, 393, 394, 395, 396, 397, 398, 399, 401, 402, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 415, 418, 419, 420, 421, 422, 423, 424, 426, 427, 428, 432, 433, 434, 435, 436, 437, 438, 440, 441, 442, 443, 444, 445, 446, 447, 450, 451, 452, 454, 455, 456, 458, 459, 460, 461, 462, 464, 465, 466, 468, 471, 472, 473, 474, 477, 479, 480, 481, 482, 483, 484, 485, 489, 490, 491, 492, 494, 495, 497, 498, 499, 500, 501, 502, 503, 505, 507, 509, 510], "45": [2, 3, 5, 7, 9, 10, 11, 14, 15, 18, 21, 22, 23, 24, 25, 26, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 54, 56, 57, 58, 59, 60, 63, 64, 65, 66, 67, 68, 69, 70, 71, 73, 74, 75, 76, 77, 79, 80, 84, 86, 88, 89, 90, 91, 92, 94, 96, 97, 99, 101, 103, 104, 105, 109, 111, 114, 115, 116, 117, 118, 119, 120, 122, 123, 124, 125, 126, 127, 128, 132, 133, 134, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 147, 148, 149, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 162, 163, 165, 166, 167, 168, 169, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 191, 192, 193, 194, 195, 196, 198, 199, 200, 201, 202, 203, 204, 206, 209, 210, 211, 212, 213, 214, 215, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 235, 236, 237, 238, 239, 241, 242, 244, 245, 248, 249, 250, 252, 253, 254, 255, 256, 257, 259, 260, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 274, 275, 277, 278, 280, 281, 282, 283, 287, 288, 289, 290, 292, 294, 296, 297, 298, 302, 303, 305, 306, 307, 308, 311, 312, 313, 314, 316, 320, 321, 322, 323, 324, 325, 327, 328, 329, 330, 332, 333, 334, 336, 338, 340, 341, 342, 344, 345, 346, 348, 349, 350, 351, 352, 354, 356, 359, 360, 361, 362, 363, 365, 366, 367, 368, 371, 372, 373, 375, 376, 377, 378, 379, 381, 382, 383, 384, 385, 386, 387, 389, 390, 391, 392, 394, 395, 398, 399, 401, 403, 404, 405, 406, 408, 411, 412, 413, 414, 416, 417, 418, 419, 421, 422, 424, 425, 426, 427, 428, 429, 431, 432, 433, 434, 435, 436, 438, 439, 440, 442, 443, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 461, 463, 466, 467, 470, 471, 472, 473, 474, 475, 478, 479, 481, 482, 484, 485, 486, 487, 488, 490, 491, 492, 493, 494, 495, 496, 498, 499, 500, 501, 502, 503, 504, 505, 507, 508, 509, 510], "46": [1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 35, 37, 38, 40, 41, 42, 43, 44, 45, 46, 48, 49, 51, 52, 53, 54, 55, 56, 57, 60, 62, 64, 65, 66, 67, 68, 69, 70, 71, 73, 76, 77, 78, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92, 93, 96, 97, 98, 99, 100, 101, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 136, 137, 138, 139, 141, 142, 143, 144, 145, 147, 148, 150, 151, 152, 154, 155, 156, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 173, 176, 177, 178, 181, 183, 185, 186, 188, 190, 191, 192, 193, 194, 195, 196, 198, 199, 200, 201, 202, 205, 206, 207, 209, 210, 211, 212, 214, 215, 218, 219, 220, 221, 222, 225, 226, 227, 228, 229, 230, 232, 234, 236, 237, 238, 239, 240, 241, 245, 246, 248, 249, 250, 253, 254, 255, 256, 257, 259, 260, 261, 262, 263, 265, 266, 269, 270, 271, 272, 274, 276, 278, 280, 281, 283, 284, 286, 287, 288, 290, 291, 292, 293, 294, 296, 297, 298, 299, 300, 301, 303, 304, 306, 308, 309, 311, 314, 315, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 328, 329, 330, 332, 334, 335, 336, 337, 338, 339, 340, 341, 343, 344, 345, 346, 347, 348, 350, 351, 352, 353, 354, 356, 357, 359, 361, 362, 363, 364, 366, 368, 369, 370, 371, 372, 373, 374, 375, 376, 378, 379, 380, 381, 383, 384, 387, 388, 389, 390, 392, 393, 396, 397, 407, 409, 410, 412, 413, 415, 416, 417, 418, 419, 420, 421, 423, 424, 425, 427, 429, 430, 431, 432, 433, 434, 435, 436, 438, 441, 443, 444, 445, 446, 447, 448, 449, 452, 454, 457, 458, 459, 461, 462, 463, 464, 465, 466, 468, 469, 470, 471, 474, 476, 477, 478, 479, 481, 482, 483, 484, 485, 486, 487, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 502, 503, 504, 506, 507, 509, 510, 511], "47": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 17, 19, 20, 21, 24, 25, 26, 27, 29, 31, 32, 33, 35, 36, 37, 39, 40, 41, 42, 43, 45, 48, 49, 50, 52, 54, 55, 56, 59, 60, 61, 64, 65, 66, 68, 69, 70, 72, 73, 75, 76, 77, 78, 79, 80, 82, 83, 84, 85, 87, 88, 89, 90, 91, 92, 93, 95, 96, 97, 98, 99, 100, 102, 103, 104, 106, 108, 109, 110, 112, 113, 116, 117, 118, 119, 120, 122, 125, 126, 127, 128, 129, 130, 131, 132, 135, 136, 137, 138, 139, 141, 142, 143, 144, 145, 146, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 159, 160, 161, 163, 164, 166, 167, 168, 169, 171, 175, 176, 178, 179, 181, 183, 184, 185, 186, 187, 188, 189, 190, 191, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 209, 210, 211, 214, 215, 217, 218, 219, 220, 221, 222, 224, 225, 226, 228, 229, 230, 231, 232, 233, 234, 235, 237, 240, 241, 242, 244, 245, 246, 247, 248, 250, 251, 253, 254, 255, 257, 260, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 275, 276, 277, 278, 280, 281, 282, 284, 286, 287, 288, 290, 292, 294, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 320, 321, 322, 323, 324, 326, 328, 329, 332, 333, 334, 335, 336, 337, 338, 340, 341, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 372, 374, 375, 376, 377, 379, 380, 381, 384, 389, 390, 391, 393, 395, 398, 399, 400, 401, 403, 404, 407, 408, 410, 412, 413, 414, 415, 416, 417, 418, 419, 422, 423, 424, 425, 427, 428, 429, 430, 431, 432, 433, 438, 439, 440, 442, 444, 445, 446, 448, 449, 450, 451, 452, 454, 455, 456, 457, 458, 460, 461, 462, 463, 464, 465, 467, 469, 470, 471, 472, 474, 475, 476, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 490, 491, 492, 493, 494, 496, 497, 498, 499, 501, 503, 504, 506, 507, 508, 509], "48": [1, 3, 5, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41, 43, 45, 46, 47, 49, 50, 51, 52, 53, 56, 58, 59, 61, 62, 63, 64, 65, 66, 68, 69, 70, 71, 75, 76, 77, 79, 80, 83, 84, 85, 86, 87, 88, 89, 90, 91, 93, 95, 96, 97, 98, 99, 100, 101, 103, 105, 106, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 120, 121, 122, 123, 124, 128, 129, 130, 131, 133, 134, 135, 137, 138, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 153, 154, 155, 156, 157, 158, 159, 160, 161, 164, 165, 167, 168, 171, 172, 174, 175, 176, 177, 179, 180, 181, 182, 183, 184, 185, 186, 188, 191, 192, 193, 196, 197, 198, 199, 200, 201, 202, 204, 205, 207, 208, 210, 211, 212, 214, 215, 216, 218, 219, 220, 221, 222, 224, 226, 228, 229, 232, 233, 235, 236, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 249, 250, 251, 252, 253, 255, 256, 257, 258, 259, 261, 262, 263, 264, 265, 266, 267, 271, 273, 274, 276, 277, 278, 279, 281, 282, 283, 284, 285, 287, 288, 289, 291, 293, 295, 296, 297, 299, 301, 302, 303, 304, 305, 306, 309, 310, 311, 312, 314, 315, 316, 317, 318, 320, 321, 322, 323, 325, 328, 332, 333, 334, 335, 337, 342, 343, 345, 346, 347, 348, 351, 352, 354, 356, 357, 358, 359, 360, 361, 362, 363, 364, 367, 368, 369, 370, 371, 372, 374, 376, 377, 378, 379, 381, 382, 383, 384, 385, 386, 387, 389, 391, 392, 393, 394, 397, 398, 399, 400, 401, 402, 403, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 417, 418, 420, 421, 422, 423, 424, 425, 426, 427, 429, 430, 433, 435, 436, 438, 439, 441, 442, 443, 444, 445, 446, 448, 450, 451, 452, 454, 456, 457, 458, 459, 460, 461, 464, 467, 468, 469, 470, 471, 472, 473, 475, 477, 478, 480, 481, 483, 484, 485, 486, 487, 488, 489, 490, 491, 494, 496, 497, 498, 499, 500, 501, 502, 503, 504, 506, 507, 508, 509, 510, 511]}}

generation_config.json

@@ -0,0 +1,12 @@
+{
+    "bos_token_id": 248044,
+    "do_sample": true,
+    "eos_token_id": [
+        248046,
+        248044
+    ],
+    "pad_token_id": 248044,
+    "temperature": 1.0,
+    "top_k": 20,
+    "top_p": 0.95
+}

model-00001-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:269ce51da510cd07333171445bdb66358afe0ff3562ae7690d0b9cc03dec625b
+size 6485594548

model-00002-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0842248b81fa762e129e5d4a8b8b68d21dfd1d8e495a20a1cbde4d96c670e92
+size 3865449064

model-00003-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:669218e54bf41c569179542f239f015cc399302d846364bcc683c45d5ee9b0f1
+size 3915287724

model-00004-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb6ba7171aa3ff7e7370b6b052dd39d24b59adacae222974d70a3a30762c87c3
+size 3755713256

model-00005-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:555dff67356ae9ad84969162024c8b9fe3ce05eb2306edde6893ad3ac88d6b0f
+size 3718942952

model-00006-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1824e38ff2b348ffc8891d0e7a646b75b73e612cef80683a3fb22c5ba6310e91
+size 3886501524

model-00007-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03f68d62c8c60ec57b6a28adfcc2290e312aa07e83448e8018cb877d2a7e149c
+size 3763275948

model-00008-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90915ae7591f5b2fb95fec8da6bce1de9fe74e96fff22d617580b25ee273bfbc
+size 3687468904

model-00009-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a18be64dcaf2102f360f6cbf27af221d065281ca680b190a94f235fae6fb12f0
+size 3747686632

model-00010-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbd528f3cdcc04561c5190f62e1879064874bd62ad5c6bdacdc9b0057a8b8082
+size 3736945628

model-00011-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ac50e83fc2f2b4d8929239beb4c1142d359b4a0836781a32153ab81562b6186
+size 3658370628

model-00012-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f719c8bb1ab3078fe738f0d6314789b78d5759274f3ea4817a874555528ed1b
+size 3686507092

model-00013-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c56871e6fd227af02f58527aaba996b1ce021b5339ffe68c2f36ceba5f3f4db8
+size 3609013092

model-00014-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be71f9a1c6e607d994de06d4c2d77f07e5a7a3fc3b7d1e63382f37729d942ab5
+size 3928684564

model-00015-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29190366c495eab7eb8d678e6106417a88cc9609e1b26022764ce1904e7b1a62
+size 4825996600

model-00016-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41bfd185d2a7256e15178e86c5a47518ab1891828959cff60f0a6d40eb0c62da
+size 4950031028

model-00017-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bf413b5ce4f99e9d8a77423bed274129b2b214c969132d2234cdb1d0b696045
+size 4950030896

model-00018-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc7ab07896b809b9d613f2b7c62f6e34921bab132a494e7b3a4e4c53901892b9
+size 4950030896

model-00019-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7992a55b72d37d4b01d36204fa47f1ac57430be44c0d9f36bc73e73a29a17968
+size 4950030904

model-00020-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b41c1e04e59961bbb0bcfd829434eca1113c5cfbbde0083a9c960d717b12d6e9
+size 4950030904

model-00021-of-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25f75d71e3eed6a2c8215cccceac480ac8f67ca7a016ac6704a4770066cd0166
+size 2700016864

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e859ca5914a2d55803b633bb7875ba48339e59b867169aedc58f497ad358f386
+size 32099062

mtp_draft/artifacts/draft_vocab_64k.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec7a9558b083f388441d4bfd33fb3d499d336febf0eb8d57ebba5032670f93cc
+size 249953

mtp_draft/bench_mtp_client.py

@@ -0,0 +1,72 @@
+"""Single-stream decode benchmark against a running vLLM server (one request at a time, wall clock per request
+incl. TTFT, temperature 0, 1024 output tokens, reasoning_effort=low, 24 prompts = code/math/chat/tool x 6 from
+prompts.json[40:46], 2 warm-up requests). tau (mean acceptance length incl. the bonus token) and per-position
+acceptance come from the server's /metrics spec-decode counters.
+usage: bench_mtp_client.py --model SERVED --out OUT.json [--port 8000]"""
+import argparse
+import json
+import re
+import time
+import urllib.request
+
+from openai import OpenAI
+
+
+def metrics(port):
+    txt = urllib.request.urlopen(f"http://localhost:{port}/metrics").read().decode()
+    m = {"drafts": 0.0, "accepted": 0.0, "per_pos": {}}
+    for line in txt.splitlines():
+        if line.startswith("vllm:spec_decode_num_drafts_total"):
+            m["drafts"] += float(line.split()[-1])
+        elif line.startswith("vllm:spec_decode_num_accepted_tokens_total"):
+            m["accepted"] += float(line.split()[-1])
+        elif line.startswith("vllm:spec_decode_num_accepted_tokens_per_pos_total"):
+            pos = int(re.search(r'position="(\d+)"', line).group(1))
+            m["per_pos"][pos] = m["per_pos"].get(pos, 0.0) + float(line.split()[-1])
+    return m
+
+
+def main():
+    p = argparse.ArgumentParser()
+    p.add_argument("--model", required=True)
+    p.add_argument("--port", default="8000")
+    p.add_argument("--per-cat", type=int, default=6)
+    p.add_argument("--max-tokens", type=int, default=1024)
+    p.add_argument("--out", required=True)
+    a = p.parse_args()
+    c = OpenAI(base_url=f"http://localhost:{a.port}/v1", api_key="x", timeout=3600)
+    prompts = json.load(open("/root/specexp/prompts.json"))
+    by = {}
+    for x in prompts:
+        by.setdefault(x["cat"], []).append(x)
+    sel = [x for cat in ["code", "math", "chat", "tool"] for x in by[cat][40:40 + a.per_cat]]
+    kw = dict(model=a.model, max_tokens=a.max_tokens, temperature=0.0,
+              extra_body={"chat_template_kwargs": {"reasoning_effort": "low"}, "ignore_eos": False})
+    for x in by["chat"][:2]:
+        c.chat.completions.create(messages=x["messages"], **kw)
+    m0 = metrics(a.port)
+    res = []
+    for x in sel:
+        t = time.perf_counter()
+        r = c.chat.completions.create(messages=x["messages"], **kw)
+        dt = time.perf_counter() - t
+        msg = r.choices[0].message
+        text = (getattr(msg, "reasoning_content", None) or getattr(msg, "reasoning", None) or "") + "\x00" + (msg.content or "")
+        res.append({"id": x["id"], "cat": x["cat"], "n": r.usage.completion_tokens, "s": dt, "text": text})
+    m1 = metrics(a.port)
+    out = {"model": a.model, "per_request": res}
+    for cat in ["code", "math", "chat", "tool", "all"]:
+        rr = [r for r in res if cat == "all" or r["cat"] == cat]
+        out[cat] = sum(r["n"] for r in rr) / sum(r["s"] for r in rr)
+    d = m1["drafts"] - m0["drafts"]
+    if d > 0:
+        out["tau"] = 1 + (m1["accepted"] - m0["accepted"]) / d
+        out["accept_per_pos"] = [(m1["per_pos"][k] - m0["per_pos"].get(k, 0.0)) / d for k in sorted(m1["per_pos"])]
+    print("RESULT", json.dumps({k: (round(v, 3) if isinstance(v, float) else
+                                    ([round(z, 3) for z in v] if isinstance(v, list) else v))
+                                for k, v in out.items() if k != "per_request"}), flush=True)
+    json.dump(out, open(a.out, "w"))
+
+
+if __name__ == "__main__":
+    main()

mtp_draft/compare_outputs.py

@@ -0,0 +1,22 @@
+"""Greedy-output agreement between benchmark runs (temperature 0): identical responses and mean shared-prefix
+fraction (characters) of each run against a reference run.
+usage: compare_outputs.py REF.json RUN.json [RUN.json ...]"""
+import json
+import os
+import sys
+
+
+def load(p):
+    return {r["id"]: r["text"] for r in json.load(open(p))["per_request"]}
+
+
+ref = load(sys.argv[1])
+for p in sys.argv[2:]:
+    run = load(p)
+    same, frac = 0, []
+    for k, a in ref.items():
+        b = run.get(k, "")
+        same += a == b
+        n = next((i for i, (x, y) in enumerate(zip(a, b)) if x != y), min(len(a), len(b)))
+        frac.append(n / max(len(a), 1))
+    print(f"{os.path.basename(p):28s} identical {same}/{len(ref)}  shared prefix {100 * sum(frac) / len(frac):5.1f}%")

mtp_draft/nvfp4_ops.py

@@ -0,0 +1,54 @@
+"""NVFP4 linear helpers built on vLLM's own kernels (same code path on B200 sm_100 and Spark sm_121).
+
+Two GEMM flavours for a bf16 weight W [N, K]:
+  * "marlin": weight-only NVFP4 (W4A16, Marlin kernel) - bf16 activations, best suited to decode GEMV;
+  * "cutlass": W4A4 (activations quantized to NVFP4 per call, CUTLASS block-scaled FP4 MMA).
+"""
+import torch
+import torch.nn as nn
+
+from vllm import _custom_ops as ops
+
+FP4_MAX, E4M3_MAX = 6.0, 448.0
+
+
+class NVFP4Linear(nn.Module):
+    def __init__(self, W: torch.Tensor, kind: str = "marlin"):
+        super().__init__()
+        assert W.dim() == 2 and W.shape[1] % 16 == 0
+        self.kind = kind
+        N, K = W.shape
+        self.output_size_per_partition, self.input_size_per_partition = N, K
+        self.params_dtype = W.dtype
+        gs = (E4M3_MAX * FP4_MAX / W.abs().amax().float().clamp(min=1e-12)).reshape(1)
+        if kind == "cutlass":
+            w4, wsf = ops.scaled_fp4_quant(W.contiguous(), gs)
+            self.register_buffer("w4", w4, persistent=False)
+            self.register_buffer("wsf", wsf, persistent=False)
+            self.register_buffer("w_gs", gs, persistent=False)
+        elif kind == "marlin":
+            from vllm.model_executor.layers.quantization.utils.marlin_utils import marlin_make_workspace_new
+            from vllm.model_executor.layers.quantization.utils.marlin_utils_fp4 import prepare_fp4_layer_for_marlin
+            w4, wsf = ops.scaled_fp4_quant(W.contiguous(), gs, is_sf_swizzled_layout=False)
+            self.weight = nn.Parameter(w4.view(torch.uint8).reshape(N, K // 2), requires_grad=False)
+            self.weight_scale = nn.Parameter(wsf.reshape(N, K // 16), requires_grad=False)
+            self.weight_global_scale = nn.Parameter((1.0 / gs).float(), requires_grad=False)
+            prepare_fp4_layer_for_marlin(self)
+            self.workspace = marlin_make_workspace_new(W.device)
+        else:
+            raise ValueError(kind)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        shp = x.shape
+        x2 = x.reshape(-1, shp[-1])
+        N = self.output_size_per_partition
+        if self.kind == "cutlass":
+            a_gs = (E4M3_MAX * FP4_MAX / x2.abs().amax().float().clamp(min=1e-12)).reshape(1)
+            a4, asf = ops.scaled_fp4_quant(x2.contiguous(), a_gs)
+            alpha = 1.0 / (a_gs * self.w_gs)
+            y = ops.cutlass_scaled_fp4_mm(a4, self.w4, asf, self.wsf, alpha, x.dtype)
+        else:
+            from vllm.model_executor.layers.quantization.utils.marlin_utils_fp4 import apply_fp4_marlin_linear
+            y = apply_fp4_marlin_linear(x2, self.weight, self.weight_scale, self.weight_global_scale,
+                                        self.workspace, N, self.input_size_per_partition)
+        return y.reshape(*shp[:-1], N)

mtp_draft/plugin/pyproject.toml

@@ -0,0 +1,15 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "q38-mtp-draft"
+version = "0.1.0"
+description = "Cheap lossless MTP drafter for Qwen3.8-Flash-Next (64k draft vocab + NVFP4 Marlin draft head / MTP linears) for vLLM 0.31 Model Runner V2"
+requires-python = ">=3.10"
+
+[project.entry-points."vllm.general_plugins"]
+q38_mtp_draft = "q38_mtp_draft:main"
+
+[tool.setuptools]
+py-modules = ["q38_mtp_draft"]

mtp_draft/plugin/q38_mtp_draft.py

@@ -0,0 +1,19 @@
+"""vLLM general plugin: install the cheap Qwen3.8-Flash-Next MTP drafter (q38_draft.py) in every vLLM process.
+Inactive unless Q38_DRAFT_VOCAB is set, so the same environment still runs unpatched servers.
+  Q38_DRAFT_HOME    directory holding q38_draft.py / nvfp4_ops.py (this repo's mtp_draft/)
+  Q38_DRAFT_VOCAB   draft vocabulary mask (.pt bool[248320]); empty -> plugin inactive
+  Q38_DRAFT_HEAD_Q  marlin (default) | cutlass | bf16      draft lm_head
+  Q38_DRAFT_MTP_Q   marlin (default) | cutlass | bf16      MTP block BF16 linears
+"""
+import os
+import sys
+
+
+def main():
+    if not os.environ.get("Q38_DRAFT_VOCAB"):
+        return
+    home = os.environ.get("Q38_DRAFT_HOME")
+    if home and home not in sys.path:
+        sys.path.insert(0, home)
+    import q38_draft
+    q38_draft.install_from_env()

mtp_draft/q38_draft.py

@@ -0,0 +1,112 @@
+"""Cheaper MTP drafter for Qwen3.8-Flash-Next (Qwen4ExpMTP) on vLLM 0.31 Model Runner V2 - lossless.
+
+Same recipe as autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK, adapted to the Flash-Next MTP layer. After
+MTPSpeculator.load_draft_model:
+  1. the drafter gets its own lm_head restricted to a draft vocabulary (64k rows gathered from the BF16 target
+     lm_head, special tokens always kept); greedy drafting takes the argmax over the kept rows and maps it back
+     to target ids (speculative_config.use_local_argmax_reduction=True routes drafting through get_top_tokens);
+  2. that head is re-quantized to NVFP4 W4A16 (Marlin GEMV);
+  3. the BF16 linear layers of the MTP block (fc_embedding, fc_hidden, attention qkv/o, shared expert) are
+     re-quantized to NVFP4 W4A16 too. The MTP routed experts are already NVFP4 in the checkpoint; the router,
+     the hyper-connection mixers and the QSA indexer stay BF16.
+Bytes read per draft step at batch 1: ~1.45 GB -> ~0.16 GB (BF16 248k x 2560 head = 1.27 GB alone).
+The target model and the lm_head it verifies with are never touched, so accepted tokens are exactly the
+target's own (greedy: identical output; sampling: same distribution).
+
+In-process use: install() before vllm.LLM(...) with VLLM_ENABLE_V1_MULTIPROCESSING=0.
+vllm serve: set Q38_DRAFT_VOCAB (and optionally Q38_DRAFT_HEAD_Q / Q38_DRAFT_MTP_Q) and put the directory that
+holds sitecustomize.py (or the plugin) on PYTHONPATH; see README.
+"""
+import os
+import sys
+import types
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+MTP_QUANT_KEYS = ("fc_embedding", "fc_hidden", "qkv_proj", "o_proj", "shared_expert")
+
+
+class _Bf16(nn.Module):
+    def __init__(self, W):
+        super().__init__()
+        self.register_buffer("w", W.contiguous(), persistent=False)
+
+    def forward(self, x):
+        return F.linear(x, self.w)
+
+
+def make_linear(W, kind):
+    if kind in (None, "bf16"):
+        return _Bf16(W)
+    from nvfp4_ops import NVFP4Linear
+    return NVFP4Linear(W, kind)
+
+
+class _QuantMethod:
+    """quant_method replacement for a vLLM linear layer (keeps the module, swaps the GEMM)."""
+
+    def __init__(self, layer, kind):
+        layer.dq = make_linear(layer.weight.data, kind)
+
+    def apply(self, layer, x, bias=None):
+        y = layer.dq(x)
+        return y + bias if bias is not None else y
+
+
+def install(vocab_path=None, head_q="marlin", mtp_q="marlin"):
+    import vllm.v1.worker.gpu.spec_decode.mtp.speculator as ms
+
+    if getattr(ms.MTPSpeculator.load_draft_model, "_q38", False):
+        return
+    keep = torch.load(vocab_path) if vocab_path else None
+    orig = ms.MTPSpeculator.load_draft_model
+
+    def load_draft_model(self, target_model, target_attn_layer_names):
+        model = orig(self, target_model, target_attn_layer_names)
+        W = model.lm_head.weight.data
+        assert W.dtype in (torch.bfloat16, torch.float16), W.dtype
+        V, dev = W.shape[0], W.device
+        idx = keep.nonzero()[:, 0].to(dev) if keep is not None else torch.arange(V, device=dev)
+        pad = (-idx.numel()) % 64            # NVFP4 GEMMs want N % 64 == 0: pad with copies of a kept id
+        if pad:
+            idx = torch.cat([idx, idx[:1].expand(pad)])
+        model.register_buffer("dh_idx", idx, persistent=False)
+        model.draft_head = make_linear(W[idx].contiguous(), head_q)
+
+        def get_top_tokens(self_, h):
+            return self_.dh_idx[self_.draft_head(h).argmax(-1)]
+
+        def compute_logits(self_, h, spec_step_idx=0):
+            lk = self_.draft_head(h).to(torch.float32)
+            out = lk.new_full((h.shape[0], V), float("-inf"))
+            out.index_copy_(1, self_.dh_idx, lk)
+            return out
+
+        model.get_top_tokens = types.MethodType(get_top_tokens, model)
+        model.compute_logits = types.MethodType(compute_logits, model)
+
+        done = []
+        if mtp_q not in (None, "bf16"):
+            for name, mod in model.model.named_modules():
+                w = getattr(mod, "weight", None)
+                if (hasattr(mod, "quant_method") and isinstance(w, torch.Tensor) and w.dim() == 2
+                        and w.dtype == torch.bfloat16 and any(k in name for k in MTP_QUANT_KEYS)
+                        and w.shape[0] % 64 == 0 and w.shape[1] % 16 == 0):
+                    mod.quant_method = _QuantMethod(mod, mtp_q)
+                    done.append(f"{name}{tuple(w.shape)}")
+        print(f"[q38-draft] vocab rows={idx.numel()} head_q={head_q} mtp_q={mtp_q} "
+              f"quantized MTP linears ({len(done)}): {done}", flush=True)
+        return model
+
+    load_draft_model._q38 = True
+    ms.MTPSpeculator.load_draft_model = load_draft_model
+
+
+def install_from_env():
+    vocab = os.environ.get("Q38_DRAFT_VOCAB")
+    if vocab:
+        install(vocab, os.environ.get("Q38_DRAFT_HEAD_Q", "marlin"), os.environ.get("Q38_DRAFT_MTP_Q", "marlin"))

mtp_draft/results/e384_k0.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 733, "s": 8.130303506040946}, {"id": "HumanEval/41", "cat": "code", "n": 440, "s": 4.539787304005586}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 1.9575089199934155}, {"id": "HumanEval/43", "cat": "code", "n": 880, "s": 8.902705920976587}, {"id": "HumanEval/44", "cat": "code", "n": 567, "s": 5.836228059022687}, {"id": "HumanEval/45", "cat": "code", "n": 167, "s": 1.9466947079636157}, {"id": "gsm8k/40", "cat": "math", "n": 310, "s": 3.419051099976059}, {"id": "gsm8k/41", "cat": "math", "n": 538, "s": 5.6530415719607845}, {"id": "gsm8k/42", "cat": "math", "n": 172, "s": 1.9405286890105344}, {"id": "gsm8k/43", "cat": "math", "n": 489, "s": 5.065521793963853}, {"id": "gsm8k/44", "cat": "math", "n": 411, "s": 4.2900476370123215}, {"id": "gsm8k/45", "cat": "math", "n": 345, "s": 3.6422453109989874}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 11.437336588976905}, {"id": "mtbench/3583619", "cat": "chat", "n": 987, "s": 11.046043527952861}, {"id": "mtbench/34377376", "cat": "chat", "n": 779, "s": 7.917029272008222}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 10.361933744046837}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 10.407953262969386}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 10.615801344974898}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 1.1472378979669884}, {"id": "glaive/41", "cat": "tool", "n": 154, "s": 2.0677673460450023}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 1.5471585509949364}, {"id": "glaive/43", "cat": "tool", "n": 75, "s": 1.2897625649929978}, {"id": "glaive/44", "cat": "tool", "n": 71, "s": 1.2427582450327463}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 1.2817979939864017}], "code": 94.72035142485548, "math": 94.3339800364686, "chat": 94.8757117592947, "tool": 62.96287478759206, "all": 92.55587206759951}

mtp_draft/results/e384_native_k3.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 729, "s": 2.8193571449955925}, {"id": "HumanEval/41", "cat": "code", "n": 414, "s": 1.816536526021082}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 0.8623655190458521}, {"id": "HumanEval/43", "cat": "code", "n": 861, "s": 3.4580993209965527}, {"id": "HumanEval/44", "cat": "code", "n": 714, "s": 2.792955293029081}, {"id": "HumanEval/45", "cat": "code", "n": 168, "s": 0.8881401250255294}, {"id": "gsm8k/40", "cat": "math", "n": 306, "s": 1.2914673539926298}, {"id": "gsm8k/41", "cat": "math", "n": 602, "s": 2.5589306599576958}, {"id": "gsm8k/42", "cat": "math", "n": 163, "s": 0.8248845480266027}, {"id": "gsm8k/43", "cat": "math", "n": 484, "s": 1.8851654370082542}, {"id": "gsm8k/44", "cat": "math", "n": 401, "s": 1.646767117024865}, {"id": "gsm8k/45", "cat": "math", "n": 346, "s": 1.4892195700085722}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 4.092011047003325}, {"id": "mtbench/3583619", "cat": "chat", "n": 988, "s": 3.5354646220221184}, {"id": "mtbench/34377376", "cat": "chat", "n": 853, "s": 3.3598524120170623}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 3.952914358000271}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 4.454029015963897}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 4.162056330998894}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 0.4963136390433647}, {"id": "glaive/41", "cat": "tool", "n": 155, "s": 0.8203358030295931}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 0.5498547999886796}, {"id": "glaive/43", "cat": "tool", "n": 75, "s": 0.45713356899796054}, {"id": "glaive/44", "cat": "tool", "n": 71, "s": 0.45059636898804456}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 0.4265658460208215}], "code": 242.53303056076578, "math": 237.40684844907742, "chat": 252.0341903005389, "tool": 169.02024356220377, "all": 241.28650946887865, "tau": 3.392622247640835, "accept_per_pos": [0.8907635115813555, 0.7955390334572491, 0.7063197026022305]}

mtp_draft/results/e384_native_k5.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 703, "s": 2.354482561990153}, {"id": "HumanEval/41", "cat": "code", "n": 444, "s": 1.7421755429822952}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 0.7637115910183638}, {"id": "HumanEval/43", "cat": "code", "n": 688, "s": 2.304368451994378}, {"id": "HumanEval/44", "cat": "code", "n": 714, "s": 2.2916528579662554}, {"id": "HumanEval/45", "cat": "code", "n": 167, "s": 0.7025055510457605}, {"id": "gsm8k/40", "cat": "math", "n": 306, "s": 1.135905829956755}, {"id": "gsm8k/41", "cat": "math", "n": 554, "s": 2.061069551971741}, {"id": "gsm8k/42", "cat": "math", "n": 171, "s": 0.7635779120028019}, {"id": "gsm8k/43", "cat": "math", "n": 461, "s": 1.5739434949937277}, {"id": "gsm8k/44", "cat": "math", "n": 408, "s": 1.3411273619858548}, {"id": "gsm8k/45", "cat": "math", "n": 342, "s": 1.1812342740013264}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 3.194308659993112}, {"id": "mtbench/3583619", "cat": "chat", "n": 977, "s": 2.977414954046253}, {"id": "mtbench/34377376", "cat": "chat", "n": 791, "s": 2.548052383994218}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 3.3816367990220897}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 4.062372940010391}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 3.7323634499916807}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 0.49537954601692036}, {"id": "glaive/41", "cat": "tool", "n": 105, "s": 0.6681176979909651}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 0.5146028829622082}, {"id": "glaive/43", "cat": "tool", "n": 75, "s": 0.556665359006729}, {"id": "glaive/44", "cat": "tool", "n": 71, "s": 0.5122391799814068}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 0.39238204003777355}], "code": 284.97189470897735, "math": 278.2722348785011, "chat": 294.73039957975766, "tool": 156.39997425682708, "all": 278.5852213651547, "tau": 4.399847386493705, "accept_per_pos": [0.8775276611980161, 0.7653567340709653, 0.6764593666539489, 0.5795497901564288, 0.5009538344143457]}

mtp_draft/results/e384_native_k7.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 712, "s": 2.2458303309977055}, {"id": "HumanEval/41", "cat": "code", "n": 397, "s": 1.3946112800040282}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 0.6491682740161195}, {"id": "HumanEval/43", "cat": "code", "n": 718, "s": 2.128362638002727}, {"id": "HumanEval/44", "cat": "code", "n": 708, "s": 1.9873309560352936}, {"id": "HumanEval/45", "cat": "code", "n": 167, "s": 0.5416151629760861}, {"id": "gsm8k/40", "cat": "math", "n": 249, "s": 0.703309926961083}, {"id": "gsm8k/41", "cat": "math", "n": 522, "s": 1.8210055900271982}, {"id": "gsm8k/42", "cat": "math", "n": 166, "s": 0.6603326629847288}, {"id": "gsm8k/43", "cat": "math", "n": 471, "s": 1.3609160670312122}, {"id": "gsm8k/44", "cat": "math", "n": 413, "s": 1.306718583975453}, {"id": "gsm8k/45", "cat": "math", "n": 346, "s": 1.0891695099999197}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 2.965524340979755}, {"id": "mtbench/3583619", "cat": "chat", "n": 982, "s": 2.8919741269783117}, {"id": "mtbench/34377376", "cat": "chat", "n": 780, "s": 2.4272121600224636}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 3.3665989469736814}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 4.0905348300002515}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 3.4870682419859804}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 0.49407145904842764}, {"id": "glaive/41", "cat": "tool", "n": 123, "s": 0.7135702630039304}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 0.46649243601132184}, {"id": "glaive/43", "cat": "tool", "n": 75, "s": 0.4326253969920799}, {"id": "glaive/44", "cat": "tool", "n": 71, "s": 0.4095555520034395}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 0.3979798569926061}], "code": 322.0103049183074, "math": 312.18250786033457, "chat": 304.64541118668404, "tool": 174.65630839656896, "all": 300.1453113965595, "tau": 5.1395244504262, "accept_per_pos": [0.8689995513683266, 0.7662628981606101, 0.6603858232391207, 0.5688649618663078, 0.4908030506953791, 0.41947061462539253, 0.36473755047106327]}

mtp_draft/results/e384_recipe64k_k3.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 729, "s": 2.727701382013038}, {"id": "HumanEval/41", "cat": "code", "n": 414, "s": 1.7476835920242593}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 0.7658775619929656}, {"id": "HumanEval/43", "cat": "code", "n": 861, "s": 3.1240575670381077}, {"id": "HumanEval/44", "cat": "code", "n": 714, "s": 2.5527657009661198}, {"id": "HumanEval/45", "cat": "code", "n": 168, "s": 0.8923461069935001}, {"id": "gsm8k/40", "cat": "math", "n": 306, "s": 1.2758885299554095}, {"id": "gsm8k/41", "cat": "math", "n": 602, "s": 2.479146101977676}, {"id": "gsm8k/42", "cat": "math", "n": 163, "s": 0.7538518089568242}, {"id": "gsm8k/43", "cat": "math", "n": 484, "s": 1.839121469995007}, {"id": "gsm8k/44", "cat": "math", "n": 401, "s": 1.5070016390527599}, {"id": "gsm8k/45", "cat": "math", "n": 346, "s": 1.3740820209495723}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 3.974660051986575}, {"id": "mtbench/3583619", "cat": "chat", "n": 988, "s": 3.468062022002414}, {"id": "mtbench/34377376", "cat": "chat", "n": 853, "s": 3.326899648993276}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 3.90042637498118}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 4.4263171890052035}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 4.085645671002567}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 0.4537901899893768}, {"id": "glaive/41", "cat": "tool", "n": 155, "s": 0.9037347360281274}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 0.5890757929882966}, {"id": "glaive/43", "cat": "tool", "n": 75, "s": 0.5211616730084643}, {"id": "glaive/44", "cat": "tool", "n": 71, "s": 0.4933487140224315}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 0.4984891830245033}], "code": 259.51633463447314, "math": 249.42866611721078, "chat": 256.10375263663406, "tool": 156.37644664054707, "all": 248.42110128744045, "tau": 3.307949790794979, "accept_per_pos": [0.8797768479776849, 0.7668061366806137, 0.6613668061366806]}

mtp_draft/results/e384_recipe64k_k5.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 703, "s": 2.5715905469842255}, {"id": "HumanEval/41", "cat": "code", "n": 444, "s": 1.5963689079508185}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 0.7399305010330863}, {"id": "HumanEval/43", "cat": "code", "n": 688, "s": 2.2338579680072144}, {"id": "HumanEval/44", "cat": "code", "n": 714, "s": 2.232825448969379}, {"id": "HumanEval/45", "cat": "code", "n": 167, "s": 0.7399766870075837}, {"id": "gsm8k/40", "cat": "math", "n": 306, "s": 1.15714238100918}, {"id": "gsm8k/41", "cat": "math", "n": 554, "s": 2.0562278060242534}, {"id": "gsm8k/42", "cat": "math", "n": 171, "s": 0.8133257809677161}, {"id": "gsm8k/43", "cat": "math", "n": 461, "s": 1.5286793949780986}, {"id": "gsm8k/44", "cat": "math", "n": 408, "s": 1.3493359020212665}, {"id": "gsm8k/45", "cat": "math", "n": 342, "s": 1.2483290410018526}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 3.1446891850209795}, {"id": "mtbench/3583619", "cat": "chat", "n": 977, "s": 2.8968855519779027}, {"id": "mtbench/34377376", "cat": "chat", "n": 791, "s": 2.4376245160237886}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 3.3674850579700433}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 4.05100225500064}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 3.5702035660506226}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 0.5177643179777078}, {"id": "glaive/41", "cat": "tool", "n": 105, "s": 0.6613505010027438}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 0.5017052160110325}, {"id": "glaive/43", "cat": "tool", "n": 75, "s": 0.4769544790033251}, {"id": "glaive/44", "cat": "tool", "n": 71, "s": 0.4288208319921978}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 0.42839869804447517}], "code": 286.2213329154901, "math": 274.98944146632175, "chat": 301.2139456421067, "tool": 162.85272635015295, "all": 282.00898588670725, "tau": 4.3014925373134325, "accept_per_pos": [0.8686567164179104, 0.75, 0.6615671641791044, 0.5537313432835821, 0.46753731343283583]}

mtp_draft/results/e384_recipe64k_k7.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 712, "s": 2.0788060189806856}, {"id": "HumanEval/41", "cat": "code", "n": 397, "s": 1.2380512969684787}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 0.5761018740013242}, {"id": "HumanEval/43", "cat": "code", "n": 718, "s": 2.004939425038174}, {"id": "HumanEval/44", "cat": "code", "n": 708, "s": 1.9182443649624474}, {"id": "HumanEval/45", "cat": "code", "n": 167, "s": 0.5227942789788358}, {"id": "gsm8k/40", "cat": "math", "n": 249, "s": 0.7487070900388062}, {"id": "gsm8k/41", "cat": "math", "n": 522, "s": 1.6979687780258246}, {"id": "gsm8k/42", "cat": "math", "n": 166, "s": 0.646616683981847}, {"id": "gsm8k/43", "cat": "math", "n": 471, "s": 1.3401818770216778}, {"id": "gsm8k/44", "cat": "math", "n": 413, "s": 1.189437868015375}, {"id": "gsm8k/45", "cat": "math", "n": 346, "s": 1.1400467989733443}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 2.917139215976931}, {"id": "mtbench/3583619", "cat": "chat", "n": 982, "s": 2.7524559579906054}, {"id": "mtbench/34377376", "cat": "chat", "n": 780, "s": 2.4050215109600686}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 3.3641807100502774}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 4.1067124850233085}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 3.5175412259995937}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 0.4751045739976689}, {"id": "glaive/41", "cat": "tool", "n": 123, "s": 0.6858198530389927}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 0.4428151000174694}, {"id": "glaive/43", "cat": "tool", "n": 75, "s": 0.41026709298603237}, {"id": "glaive/44", "cat": "tool", "n": 71, "s": 0.3740595959825441}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 0.4027617870015092}], "code": 345.4876695366444, "math": 320.4218699568157, "chat": 307.29603395733346, "tool": 182.3831491759556, "all": 308.8827079576811, "tau": 4.926514825956167, "accept_per_pos": [0.8620541469703481, 0.7494628276751182, 0.6355822948001719, 0.5371723248818221, 0.45423291792006876, 0.3807477438762355, 0.30726256983240224]}

mtp_draft/results/e384_recipe64k_k9.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 726, "s": 2.2813628219882958}, {"id": "HumanEval/41", "cat": "code", "n": 395, "s": 1.5880196589860134}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 0.7044576379703358}, {"id": "HumanEval/43", "cat": "code", "n": 731, "s": 2.2817975389771163}, {"id": "HumanEval/44", "cat": "code", "n": 711, "s": 2.0792866839910857}, {"id": "HumanEval/45", "cat": "code", "n": 167, "s": 0.5578156499541365}, {"id": "gsm8k/40", "cat": "math", "n": 194, "s": 0.5612203259952366}, {"id": "gsm8k/41", "cat": "math", "n": 525, "s": 1.8817650109995157}, {"id": "gsm8k/42", "cat": "math", "n": 172, "s": 0.6732398690073751}, {"id": "gsm8k/43", "cat": "math", "n": 481, "s": 1.4211978020030074}, {"id": "gsm8k/44", "cat": "math", "n": 461, "s": 1.2046463890001178}, {"id": "gsm8k/45", "cat": "math", "n": 352, "s": 1.0732165930094197}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 3.4876003169920295}, {"id": "mtbench/3583619", "cat": "chat", "n": 1024, "s": 3.0580359949963167}, {"id": "mtbench/34377376", "cat": "chat", "n": 781, "s": 2.506533487990964}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 2.8171972099808045}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 4.456988577963784}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 3.5919092850526795}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 0.42223657004069537}, {"id": "glaive/41", "cat": "tool", "n": 171, "s": 0.7977454219944775}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 0.39193789300043136}, {"id": "glaive/43", "cat": "tool", "n": 76, "s": 0.36931639799149707}, {"id": "glaive/44", "cat": "tool", "n": 68, "s": 0.3202864989871159}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 0.3118133860407397}], "code": 306.44471485496496, "math": 320.6028335716688, "chat": 296.26074548320616, "tool": 212.37221861628046, "all": 297.37669708275394, "tau": 5.309665597801191, "accept_per_pos": [0.8625744388456252, 0.7530920751259734, 0.6284928996793404, 0.5231333027943198, 0.4406779661016949, 0.3673843334860284, 0.3005038937242327, 0.24049473202015575, 0.19331195602382042]}

mtp_draft/results/orig_native_k5.json

@@ -0,0 +1 @@
+{"model": "bench", "per_request": [{"id": "HumanEval/40", "cat": "code", "n": 689, "s": 2.362393995979801}, {"id": "HumanEval/41", "cat": "code", "n": 495, "s": 1.9359628790407442}, {"id": "HumanEval/42", "cat": "code", "n": 179, "s": 0.6903294100193307}, {"id": "HumanEval/43", "cat": "code", "n": 944, "s": 3.16420370596461}, {"id": "HumanEval/44", "cat": "code", "n": 425, "s": 1.4295662239892408}, {"id": "HumanEval/45", "cat": "code", "n": 172, "s": 0.7223699850146659}, {"id": "gsm8k/40", "cat": "math", "n": 258, "s": 0.9329101420007646}, {"id": "gsm8k/41", "cat": "math", "n": 541, "s": 1.9815324809751473}, {"id": "gsm8k/42", "cat": "math", "n": 163, "s": 0.7296427360270172}, {"id": "gsm8k/43", "cat": "math", "n": 479, "s": 1.7465769789996557}, {"id": "gsm8k/44", "cat": "math", "n": 397, "s": 1.2734857919858769}, {"id": "gsm8k/45", "cat": "math", "n": 396, "s": 1.328218968003057}, {"id": "mtbench/56561594", "cat": "chat", "n": 1024, "s": 3.39288957597455}, {"id": "mtbench/3583619", "cat": "chat", "n": 979, "s": 2.9747322580078617}, {"id": "mtbench/34377376", "cat": "chat", "n": 833, "s": 2.504794050997589}, {"id": "mtbench/46306085", "cat": "chat", "n": 1024, "s": 3.21053582202876}, {"id": "mtbench/50530142", "cat": "chat", "n": 1024, "s": 4.134806908958126}, {"id": "mtbench/65906068", "cat": "chat", "n": 1024, "s": 3.8171016260166653}, {"id": "glaive/40", "cat": "tool", "n": 71, "s": 0.5119000870035961}, {"id": "glaive/41", "cat": "tool", "n": 174, "s": 0.8766445890069008}, {"id": "glaive/42", "cat": "tool", "n": 98, "s": 0.5194328450015746}, {"id": "glaive/43", "cat": "tool", "n": 75, "s": 0.4582692759577185}, {"id": "glaive/44", "cat": "tool", "n": 71, "s": 0.44306598301045597}, {"id": "glaive/45", "cat": "tool", "n": 71, "s": 0.41111837298376486}], "code": 281.80970194311817, "math": 279.5166904384815, "chat": 294.88601011648876, "tool": 173.8897599113674, "all": 279.3094103941716, "tau": 4.494399382000772, "accept_per_pos": [0.8895326380842024, 0.7864040169949787, 0.6925453843182696, 0.601004248744689, 0.5249130938586327]}

mtp_draft/run_mtp_serve.sh

@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+# One benchmark configuration: start vLLM serve, run the single-stream client, stop the server.
+# usage: run_mtp_serve.sh MODEL_VIEW TAG K [DRAFT_VOCAB]
+M=$1; TAG=$2; K=$3; V=${4:-}
+cd /root/q38prune
+mkdir -p results/mtp
+while [ "$(awk '$1=="shmem"{print $2}' /sys/fs/cgroup/memory.stat)" -gt 2000000000 ]; do sleep 10; done
+SPEC=()
+if [ "$K" -gt 0 ]; then
+  if [ -n "$V" ]; then
+    export Q38_DRAFT_VOCAB=$V Q38_DRAFT_HEAD_Q=marlin Q38_DRAFT_MTP_Q=marlin
+    SPEC=(--speculative-config "{\"method\":\"mtp\",\"num_speculative_tokens\":$K,\"use_local_argmax_reduction\":true}")
+  else
+    unset Q38_DRAFT_VOCAB
+    SPEC=(--speculative-config "{\"method\":\"mtp\",\"num_speculative_tokens\":$K}")
+  fi
+fi
+./serve_vllm.sh "$M" bench 8000 --kv-cache-dtype auto --max-model-len 16384 --max-num-seqs 8 --block-size 48 \
+  --no-enable-prefix-caching --limit-mm-per-prompt '{"image":0,"video":0}' "${SPEC[@]}" > logs/mtp_serve_$TAG.log 2>&1 &
+SP=$!
+for i in $(seq 1 120); do curl -s localhost:8000/v1/models > /dev/null && break; sleep 10; done
+/root/wdm/envs/jev/bin/python draft/bench_mtp_client.py --model bench --out results/mtp/$TAG.json > logs/mtp_client_$TAG.log 2>&1
+grep RESULT logs/mtp_client_$TAG.log | cut -c1-600
+grep -o "\[q38-draft\].*" logs/mtp_serve_$TAG.log | head -1 | cut -c1-400
+kill $(pgrep -f "served-model-name benc[h]") 2>/dev/null
+wait $SP 2>/dev/null
+sleep 15

preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+    "size": {
+        "longest_edge": 16777216,
+        "shortest_edge": 65536
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "image_processor_type": "Qwen2VLImageProcessorFast"
+}

serve_vllm.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+# Serve this complete checkpoint with the vLLM layout patch shipped alongside it.
+set -euo pipefail
+ROOT=$(cd "$(dirname "$0")" && pwd)
+export PYTHONPATH="$ROOT/vllm_patch${PYTHONPATH:+:$PYTHONPATH}"
+export VLLM_WORKER_MULTIPROC_METHOD=spawn
+export VLLM_SSM_CONV_STATE_LAYOUT=DS
+if [ -d /usr/lib/x86_64-linux-gnu ]; then
+  export LD_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+fi
+exec "${VLLM:-vllm}" serve "$ROOT" --served-model-name "${SERVED_NAME:-e384-rl}" \
+  --port "${PORT:-8000}" --tensor-parallel-size 1 --dtype bfloat16 \
+  --kv-cache-dtype fp8 --quantization modelopt_mixed --mamba-cache-mode align \
+  --enable-prefix-caching --enable-chunked-prefill --max-model-len 72000 \
+  --max-num-seqs 128 --gpu-memory-utilization 0.90 --reasoning-parser qwen3 \
+  --tool-call-parser qwen3_xml --enable-auto-tool-choice "$@"