Add MI300X row-scaled FP8 Inkling 2 checkpoint with vision and DSpark

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -0,0 +1,118 @@
+---
+license: apache-2.0
+base_model: thinkingmachines/Inkling-2-Small-MXFP8
+base_model_relation: quantized
+library_name: vllm
+pipeline_tag: image-text-to-text
+tags:
+- fp8
+- rocm
+- mi300x
+- inkling2
+- moe
+- dspark
+- vision
+---
+
+# Inkling 2 Small FP8 for AMD MI300X
+
+A row-scaled E4M3FNUZ conversion of
+[Thinking Machines' Inkling 2 Small MXFP8](https://huggingface.co/thinkingmachines/Inkling-2-Small-MXFP8),
+revision `3febc62259ee92ea36e2fb6e8303371c4eb3922c`.
+The original model was developed by Thinking Machines Lab. This repository
+contains a precision conversion, not a new trained model. The upstream model
+card is preserved in [UPSTREAM_README.md](UPSTREAM_README.md).
+
+## Precision and files
+
+The 96 expert projection tensors use native AMD **E4M3FNUZ FP8**, with one FP32
+scale per expert output channel. Activations use dynamic per-token E4M3FNUZ
+quantization. Weights are dequantized from the original MXFP8 block scales and
+requantized with `scale = max(abs(row)) / 240`. Attention, dense projections,
+vision, and the DSpark draft retain the upstream precision. This is mixed
+precision inference.
+
+All 48 target weight shards, tokenizer/configuration files, vision weights,
+and `mtp.safetensors` are included. The checkpoint is approximately 605.14 GB.
+[conversion_manifest.json](conversion_manifest.json) records the source revision,
+per-shard tensor sizes, and serialization verification fingerprints.
+The original checkpoint is unchanged.
+
+`hf_quant_config.json` marks converted experts with `compute_dtype: fp8`;
+`config.json` selects `expert_fp8_scale_mode: row`. Expert scale tensors retain
+the `_weight_scale` suffix and have shape `[384, output_channels]`, dtype FP32.
+The loader reads the converted weights directly without requantizing them.
+
+## Serving
+
+Use the ROCm vLLM image built by the Seekr GitLab pipeline from the source tree
+containing Inkling 2 and prequantized row-FP8 support. The Docker build validates
+that the wheel and installed package contain the current Inkling implementation,
+renderer, and model registrations. No separate patch is distributed here.
+This checkpoint requires that implementation; an unmodified vLLM installation
+is not a supported loader for this format.
+
+Validated hardware: **8 AMD MI300X GPUs (gfx942), 192 GiB per GPU**.
+Other GPU architectures have not been validated.
+
+Inside the deployment image:
+
+```bash
+export OMP_NUM_THREADS=1
+export VLLM_ROCM_USE_AITER=1
+export VLLM_ROCM_USE_AITER_CUSTOM_AR=1
+
+vllm serve seekr-org/Inkling-2-Small-FP8-ROCm \
+  --quantization inkling2 --tensor-parallel-size 8 \
+  --max-model-len 262144 --max-num-batched-tokens 2048 --max-num-seqs 8 \
+  --no-enable-prefix-caching \
+  --reasoning-parser inkling --limit-mm-per-prompt '{"image":2}' \
+  --gpu-memory-utilization 0.9 \
+  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}' \
+  --compilation-config '{"mode":0,"cudagraph_mode":"FULL_DECODE_ONLY","decode_cudagraph_capture_request_sizes":[1,2,4,8],"use_breakable_cudagraph":false}' \
+  --kernel-config '{"enable_jit_warmup":false}'
+```
+
+The image's entrypoint is `vllm serve`. With Docker, expose `/dev/kfd` and
+`/dev/dri`, use host IPC, mount a persistent Hugging Face cache, and pass the
+model name and flags above as container arguments. Set `HF_TOKEN` in the
+container environment if required; no credentials are included in this repo.
+
+Images use standard `image_url` chat content parts. To request direct answers,
+set `chat_template_kwargs.reasoning_effort` to `0`. The default permits reasoning.
+
+## Measurements and accuracy
+
+Prior end-to-end measurements of this row-FP8 quantization used 24 distinct
+math/programming/prose tasks, about 2K input and 512 completion tokens per
+request, temperature 1, default reasoning, prefix caching disabled, and the
+seven-token probabilistic DSpark configuration above:
+
+| Metric | Row FP8 | Original MXFP8 scales |
+|---|---:|---:|
+| C1 output tokens/s | 247.47 | 141.20 |
+| C8 aggregate output tokens/s | 800.03 | 348.82 |
+| GSM8K, fixed 50-problem sample | 48/50 | 48/50 |
+| HumanEval, fixed 20-problem sample | 19/20 | 20/20 |
+
+Throughput includes prefill, the final partially occupied batch, and reasoning
+tokens. These are short-prompt results, not throughput at 256K input. The C8
+1,000-token/s goal was not reached. Prose alone averaged 198.89 tokens/s at C1.
+
+The paired accuracy check used temperature 0, reasoning effort 0, and a
+2,048-token generation limit for each mode. The row-FP8 coding miss exhausted
+its output budget. One GSM8K reference answer is arithmetically incorrect;
+raw scores preserve that label. These small samples do not establish full
+benchmark parity, and greedy continuations can vary with batching.
+
+Both quantization paths passed a 261,040-input-token recall check and text,
+image colors, OCR, multiple-image, and streaming checks before serialization.
+The exported checkpoint's additional load validation is recorded in
+[validation_results.json](validation_results.json).
+
+## License and attribution
+
+Apache License 2.0, inherited from the upstream checkpoint. See [LICENSE](LICENSE).
+Original model: Thinking Machines Lab. This repository converts the expert
+precision for AMD MI300X deployment and retains the upstream model files and
+attribution. Model purpose and upstream information are in UPSTREAM_README.md.

chat_template.jinja

@@ -0,0 +1,126 @@
+{%- set effort_map = {"none": 0.0, "minimal": 0.1, "low": 0.2, "medium": 0.7, "high": 0.9, "xhigh": 0.99, "max": 0.99} -%}
+{%- set role_token = {"user": "<|message_user|>", "assistant": "<|message_model|>", "system": "<|message_system|>", "tool": "<|message_tool|>"} -%}
+
+{%- macro emit_thinking_effort() -%}
+    {%- set eff = reasoning_effort if reasoning_effort is defined and reasoning_effort is not none else 0.9 -%}
+    {%- if eff is string -%}
+        {%- set key = eff | trim -%}
+        {%- if key not in effort_map -%}
+            {{- raise_exception("Unknown reasoning_effort: " ~ eff) -}}
+        {%- endif -%}
+        {%- set num = effort_map[key] -%}
+    {%- else -%}
+        {%- set num = eff | float -%}
+    {%- endif -%}
+    {%- if num < 0.0 or num > 0.99 -%}
+        {{- raise_exception("reasoning_effort must be in [0.0, 0.99]") -}}
+    {%- endif -%}
+    {{- "<|message_system|><|content_text|>Thinking effort level: " -}}
+    {%- if num == 0.0 -%}0{%- else -%}{{ num }}{%- endif -%}
+    {{- "<|end_message|>" -}}
+{%- endmacro -%}
+
+{%- if tools -%}
+    {%- set tool_state = namespace(specs=[]) -%}
+    {%- for tool in tools -%}
+        {%- set fn = tool.function if tool.function is defined else tool -%}
+        {%- set spec = {
+            "description": (fn.description if fn.description is defined and fn.description else ""),
+            "name": fn.name,
+            "parameters": (fn.parameters if fn.parameters is defined and fn.parameters else {}),
+            "type": (tool.type if tool.type is defined and tool.type else "function"),
+        } -%}
+        {%- set tool_state.specs = tool_state.specs + [spec] -%}
+    {%- endfor -%}
+    {{- "<|message_system|>tool_declare<|content_xml|>" -}}
+    {{- tool_state.specs | tojson(sort_keys=true, separators=(",", ":")) -}}
+    {{- "<|end_message|>" -}}
+{%- endif -%}
+
+{%- set state = namespace(effort_emitted=false) -%}
+{%- for message in messages -%}
+    {%- if message.role not in role_token -%}
+        {{- raise_exception("Unknown message role: " ~ message.role) -}}
+    {%- endif -%}
+    {%- if not state.effort_emitted and message.role != "system" -%}
+        {{- emit_thinking_effort() -}}
+        {%- set state.effort_emitted = true -%}
+    {%- endif -%}
+
+    {%- set header = namespace(text=role_token[message.role]) -%}
+    {%- if message.role == "tool" -%}
+        {%- if message.name is defined and message.name -%}
+            {%- set header.text = header.text ~ message.name -%}
+        {%- elif message.tool_call_id is defined and message.tool_call_id -%}
+            {%- for prev in messages -%}
+                {%- if prev.role == "assistant" and prev.tool_calls -%}
+                    {%- for tc in prev.tool_calls -%}
+                        {%- if tc.id is defined and tc.id == message.tool_call_id and tc.function.name is defined -%}
+                            {%- set header.text = header.text ~ tc.function.name -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+    {%- elif message.role == "assistant" and message.name is defined and message.name -%}
+        {%- set header.text = header.text ~ message.name -%}
+    {%- endif -%}
+    {%- set rtok = header.text -%}
+
+    {%- if message.role == "assistant" and message.reasoning_content is defined and message.reasoning_content -%}
+        {{- rtok ~ "<|content_thinking|>" ~ message.reasoning_content ~ "<|end_message|>" -}}
+    {%- endif -%}
+
+    {%- if message.content is string and (message.content or not (message.role == "assistant" and (message.tool_calls or message.reasoning_content))) -%}
+        {{- rtok ~ "<|content_text|>" ~ message.content ~ "<|end_message|>" -}}
+    {%- elif message.content -%}
+        {%- for part in message.content -%}
+            {%- if part is string -%}
+                {{- rtok ~ "<|content_text|>" ~ part ~ "<|end_message|>" -}}
+            {%- elif part.type is not defined or part.type in ("text", "input_text") -%}
+                {%- set text_part = (part.text if part.text is defined and part.text is string else "") -%}
+                {{- rtok ~ "<|content_text|>" ~ text_part ~ "<|end_message|>" -}}
+            {%- elif part.type in ("image", "input_image", "image_url") -%}
+                {%- if message.role not in ("user", "tool") -%}
+                    {{- raise_exception("Image content is only supported for user and tool messages") -}}
+                {%- endif -%}
+                {{- rtok ~ "<|content_image|><|unused_200054|><|end_message|>" -}}
+            {%- elif part.type in ("audio", "input_audio", "audio_url") -%}
+                {{- raise_exception("Inkling 2 does not accept audio input") -}}
+            {%- else -%}
+                {{- raise_exception("Unsupported content part type: " ~ part.type) -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endif -%}
+
+    {%- if message.role == "assistant" and message.tool_calls -%}
+        {%- for tc in message.tool_calls -%}
+            {%- set fn = tc.function -%}
+            {%- if fn.name is not defined or fn.name is not string -%}
+                {{- raise_exception("tool call function name must be a string") -}}
+            {%- endif -%}
+            {%- if message.name is defined and message.name and message.name != fn.name -%}
+                {{- raise_exception("assistant name " ~ message.name ~ " does not match its tool call " ~ fn.name) -}}
+            {%- endif -%}
+            {%- set args = fn.arguments if fn.arguments is defined and fn.arguments else {} -%}
+            {%- if args is string -%}
+                {{- raise_exception("tool call arguments must be a parsed object, not a JSON string; canonicalize upstream") -}}
+            {%- endif -%}
+            {%- if args is not mapping -%}
+                {{- raise_exception("tool call arguments must be an object") -}}
+            {%- endif -%}
+            {{- "<|message_model|>" ~ fn.name ~ "<|content_invoke_tool_json|>" -}}
+            {{- '{"name":' ~ (fn.name | tojson(sort_keys=true, separators=(",", ":"))) ~ ',"args":' -}}
+            {{- (args | tojson(sort_keys=true, separators=(",", ":"))) -}}
+            {{- "}<|end_message|>" -}}
+        {%- endfor -%}
+    {%- endif -%}
+
+    {%- if message.role == "assistant" -%}
+        {{- "<|content_model_end_sampling|>" -}}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if not state.effort_emitted -%}
+    {{- emit_thinking_effort() -}}
+{%- endif -%}

config.json

@@ -0,0 +1,137 @@
+{
+  "architectures": [
+    "Inkling2ForConditionalGeneration"
+  ],
+  "model_type": "inkling2",
+  "eos_token_id": 200006,
+  "text_config": {
+    "model_max_length": 1048576,
+    "torch_dtype": "bfloat16",
+    "hidden_size": 4608,
+    "num_hidden_layers": 48,
+    "vocab_size": 201024,
+    "num_attention_heads": 48,
+    "num_key_value_heads": 8,
+    "head_dim": 128,
+    "d_rel": 16,
+    "rel_extent": 1024,
+    "q_bias": false,
+    "o_bias": false,
+    "log_scaling_n_floor": 128000,
+    "log_scaling_alpha": 0.1,
+    "rms_norm_eps": 1e-06,
+    "pre_norm_pre_affine": false,
+    "final_norm_pre_affine": false,
+    "use_value_norm_no_gamma": true,
+    "qk_l2_clip_threshold": 22.627416997969522,
+    "rel_l2_clip_threshold": 8.0,
+    "local_layer_ids": [
+      0,
+      1,
+      2,
+      3,
+      4,
+      6,
+      7,
+      8,
+      9,
+      10,
+      12,
+      13,
+      14,
+      15,
+      16,
+      18,
+      19,
+      20,
+      21,
+      22,
+      24,
+      25,
+      26,
+      27,
+      28,
+      30,
+      31,
+      32,
+      33,
+      34,
+      36,
+      37,
+      38,
+      39,
+      40,
+      42,
+      43,
+      44,
+      45,
+      46
+    ],
+    "sconv_kernel_size": 4,
+    "unpadded_vocab_size": 200058,
+    "final_logit_softcapping": 30.0,
+    "swa_head_dim": 128,
+    "swa_num_attention_heads": 48,
+    "swa_num_key_value_heads": 8,
+    "sliding_window_size": 1024,
+    "n_layers_per_attn_res_block": 6,
+    "attn_res_normalize_values": true,
+    "n_routed_experts": 384,
+    "num_experts_per_tok": 8,
+    "n_shared_experts": 0,
+    "intermediate_size": 2304,
+    "mlp_gate_softcap": 4.0,
+    "mlp_up_softcap": 25.0,
+    "route_scale": 8.0,
+    "use_gate_bias": true,
+    "expert_fp8_scale_mode": "row"
+  },
+  "vision_config": {
+    "temporal_patch_size": 2,
+    "native_patch_size": 20,
+    "adapter_merge_size": 2,
+    "projection_hidden_dim": 4096,
+    "projector_bias": false,
+    "hidden_size": 1024,
+    "num_hidden_layers": 23,
+    "num_attention_heads": 16,
+    "mlp_ratio": 4.0,
+    "posemb_grid_size": 32,
+    "layer_norm_eps": 1e-05
+  },
+  "mtp_config": {
+    "hidden_size": 4608,
+    "num_hidden_layers": 5,
+    "num_attention_heads": 48,
+    "num_key_value_heads": 8,
+    "head_dim": 128,
+    "intermediate_size": 18432,
+    "max_position_embeddings": 1048576,
+    "torch_dtype": "bfloat16",
+    "q_bias": false,
+    "o_bias": false,
+    "rms_norm_eps": 1e-06,
+    "rope_theta": 1000000.0,
+    "local_layer_ids": [
+      0,
+      1,
+      3,
+      4
+    ],
+    "sliding_window_size": 1024,
+    "block_size": 8,
+    "markov_rank": 256,
+    "markov_head_type": "rnn_hidden",
+    "target_layer_ids": [
+      5,
+      11,
+      17,
+      23,
+      29,
+      35,
+      41,
+      48
+    ]
+  },
+  "expert_fp8_scale_mode": "row"
+}

conversion_manifest.json

@@ -0,0 +1,590 @@
+{
+  "source_model": "thinkingmachines/Inkling-2-Small-MXFP8",
+  "source_revision": "3febc62259ee92ea36e2fb6e8303371c4eb3922c",
+  "target_model": "seekr-org/Inkling-2-Small-FP8-ROCm",
+  "expert_weight_dtype": "float8_e4m3fnuz",
+  "expert_scale_dtype": "float32",
+  "expert_scale_granularity": "per_output_channel",
+  "activation_quantization": "dynamic_per_token_e4m3fnuz",
+  "expert_weight_tensors": 96,
+  "nonexpert_weights": "unchanged",
+  "mtp": "unchanged",
+  "shards": [
+    {
+      "file": "model-00001-of-00048.safetensors",
+      "bytes": 14495803536,
+      "tensor_bytes": 14495789568,
+      "tensor_count": 134,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.0.mlp.experts.w13_weight": "5d4b6e27051faebb9a5f67cbd47c459d0edf39d049d4ccfdc62639851059e0d4",
+        "model.llm.layers.0.mlp.experts.w2_weight": "11a430aad28e66b35dfde80ee61483b4ae9de2031b0195fe82a8c8627273ec96"
+      },
+      "seconds": 15.63
+    },
+    {
+      "file": "model-00002-of-00048.safetensors",
+      "bytes": 12444708336,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.1.mlp.experts.w13_weight": "574a19b6b2cda6bc274696b715a5c227ae80324542e53cbc29411b80a120a4e2",
+        "model.llm.layers.1.mlp.experts.w2_weight": "6f7a7e140a19b6841e72330ffc39435f32a70d3636c776c8fac499b5430604c6"
+      },
+      "seconds": 14.81
+    },
+    {
+      "file": "model-00003-of-00048.safetensors",
+      "bytes": 12445298408,
+      "tensor_bytes": 12445295616,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.10.mlp.experts.w13_weight": "6d591b032fdf5a560b760ba0160f3c2200b17cb96fff8f7e758aaf32fae4376f",
+        "model.llm.layers.10.mlp.experts.w2_weight": "c3e868d78e40a91fabcc7c8040744d1fb7c58d91fb719b3e7a53f7555b954095"
+      },
+      "seconds": 15.41
+    },
+    {
+      "file": "model-00004-of-00048.safetensors",
+      "bytes": 12444727000,
+      "tensor_bytes": 12444724224,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.11.mlp.experts.w13_weight": "c2dc4be8a79f0821071987712c326e5504c9b5c6307eef0a5c571a2dd9839a24",
+        "model.llm.layers.11.mlp.experts.w2_weight": "499d482500e0ee1bc4c204a132f0227b1fe14869d9b606e75dee38bed621a675"
+      },
+      "seconds": 15.5
+    },
+    {
+      "file": "model-00005-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.12.mlp.experts.w13_weight": "b876b06a103f692e0f6b0bb2d5928a4ede00b696892d197a3758592781598c1c",
+        "model.llm.layers.12.mlp.experts.w2_weight": "9d47dc0049df162aa190c0d33439e0116bbf8c5205a9377748f5da1b661cae77"
+      },
+      "seconds": 14.33
+    },
+    {
+      "file": "model-00006-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.13.mlp.experts.w13_weight": "cfc5533a84445ddb8597e051e981938ff4e8b989ae168f7a31890ba8d973d842",
+        "model.llm.layers.13.mlp.experts.w2_weight": "c51748e0ea108441b1a1b428f12bc02d173ff09efb1031fb31dd8f3391d4420a"
+      },
+      "seconds": 14.91
+    },
+    {
+      "file": "model-00007-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.14.mlp.experts.w13_weight": "a96ffb4a5c3c493c091db1a0243884ded5a2ed86fc36b872bf197adc2ced1287",
+        "model.llm.layers.14.mlp.experts.w2_weight": "ddb178a8b1f723d7393896744a9582941661caa0f0102c15c1e209fd1bc7ffc6"
+      },
+      "seconds": 14.49
+    },
+    {
+      "file": "model-00008-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.15.mlp.experts.w13_weight": "5b555aa15581b7dca3c7685e56b31ccc928de9e13f1e13aeb111c79abf3d9f74",
+        "model.llm.layers.15.mlp.experts.w2_weight": "9d7cedd75ab25a305bc663dc8163316fef34e79e1e26a73d30865757b024d5e1"
+      },
+      "seconds": 14.83
+    },
+    {
+      "file": "model-00009-of-00048.safetensors",
+      "bytes": 12445298408,
+      "tensor_bytes": 12445295616,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.16.mlp.experts.w13_weight": "20c4b32e41bf42a66bf82d7527189ba77f7556651f2b49beed6b48b7821e55a8",
+        "model.llm.layers.16.mlp.experts.w2_weight": "57420a13956bf4c6422ec8d6b78ff8d25632d67247e83ecf374b59ebd6fca39d"
+      },
+      "seconds": 14.42
+    },
+    {
+      "file": "model-00010-of-00048.safetensors",
+      "bytes": 12444727000,
+      "tensor_bytes": 12444724224,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.17.mlp.experts.w13_weight": "04f5117f7f6103870ae5f18ca38d20cfa9222972387acea9fb8b53af8a9f43a6",
+        "model.llm.layers.17.mlp.experts.w2_weight": "7e0ec280bb5e062997eae7dc9023a6aa19491c909c03b1a9d75b4ce4c6961ad1"
+      },
+      "seconds": 14.38
+    },
+    {
+      "file": "model-00011-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.18.mlp.experts.w13_weight": "bf7f84e49d7c27f1d2258b5d5cdb6cd97d29c1c723962c0ddd5c93b53076db97",
+        "model.llm.layers.18.mlp.experts.w2_weight": "2736efab8b49e29dda571f16a3cab3cd8b3ee284ece5251c9af568f512192e4c"
+      },
+      "seconds": 14.38
+    },
+    {
+      "file": "model-00012-of-00048.safetensors",
+      "bytes": 12444708336,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.19.mlp.experts.w13_weight": "9c55a3eb0bf98133ce6af3baa84ee04189f00792aae058049c412a4cdff499ca",
+        "model.llm.layers.19.mlp.experts.w2_weight": "49ec17c2d207fb3f4979a06e668a67f4bf1c90742235ed2af7cd5dd6b73db326"
+      },
+      "seconds": 15.2
+    },
+    {
+      "file": "model-00013-of-00048.safetensors",
+      "bytes": 12444708336,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.2.mlp.experts.w13_weight": "fb5eb9254ac6714dc634428c368631a9c84311e9da5f1ba9b87e96dc81e6b4b1",
+        "model.llm.layers.2.mlp.experts.w2_weight": "872aa40c2650c00b02ec0a7639ef17bcc6ae9230a42630a4781054c4af2ee764"
+      },
+      "seconds": 15.64
+    },
+    {
+      "file": "model-00014-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.20.mlp.experts.w13_weight": "64f3e552f7b7c67eba7455d63b6fdcce6c6eb385f685a88b7af33713c81b1948",
+        "model.llm.layers.20.mlp.experts.w2_weight": "0a16b5571fd61226a7af229c5c26c5f90ba2a87c96d7d7a25679e7476fa0f2ce"
+      },
+      "seconds": 14.21
+    },
+    {
+      "file": "model-00015-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.21.mlp.experts.w13_weight": "3e8499f011dc7cb11e589cd93b1ced106ea1c47a815f18e6730444a4e1f0dce2",
+        "model.llm.layers.21.mlp.experts.w2_weight": "a0cc89c7f66b6b4f20e525d3bf30306a3600a6a4dc24ae8d34a2fff0b4eb8cf1"
+      },
+      "seconds": 14.75
+    },
+    {
+      "file": "model-00016-of-00048.safetensors",
+      "bytes": 12445298408,
+      "tensor_bytes": 12445295616,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.22.mlp.experts.w13_weight": "959ec58cd222a2ac72bf8a101a1e964fdc26ac67af8bad3640507e4ddc7e0bfe",
+        "model.llm.layers.22.mlp.experts.w2_weight": "c5a1421fb2f9780655bef41a6e00e7c1757db938db5f3fb938db2e6e13278812"
+      },
+      "seconds": 14.78
+    },
+    {
+      "file": "model-00017-of-00048.safetensors",
+      "bytes": 12444727000,
+      "tensor_bytes": 12444724224,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.23.mlp.experts.w13_weight": "79def23507fea85c233a3fdc6da0888cbd191e984360c451ff66192bb1573e87",
+        "model.llm.layers.23.mlp.experts.w2_weight": "304441c3f66223d0f2eaa168bfed7d28cff7a5668362eeb35eb24786c846d6b0"
+      },
+      "seconds": 15.89
+    },
+    {
+      "file": "model-00018-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.24.mlp.experts.w13_weight": "a66a1f461bc234f2e3e6ef42b7f66a40faeecffc43021cfac80e6ae3a33796ec",
+        "model.llm.layers.24.mlp.experts.w2_weight": "b6a1f14c849c71d3fbb7e3fa30827a6db6aec3d81a113c1929249316c69153dd"
+      },
+      "seconds": 14.88
+    },
+    {
+      "file": "model-00019-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.25.mlp.experts.w13_weight": "6ffe5b43166f8998f8a56ec87a44ebc79a893f6e84654ee81f8b03be58184a14",
+        "model.llm.layers.25.mlp.experts.w2_weight": "d87a5ce5f9ee8e73cd5ed27ab93ae6dc166e88969473832b470d948c138d55db"
+      },
+      "seconds": 14.91
+    },
+    {
+      "file": "model-00020-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.26.mlp.experts.w13_weight": "6658ffbc9ce9b4b384f6a56747d96ad545ae5e3dced1041647cf6560df7a7d94",
+        "model.llm.layers.26.mlp.experts.w2_weight": "f38a5d4329ec4bf9aa3a3665490dc92785774ba426034531966abbb1ca362fa0"
+      },
+      "seconds": 15.28
+    },
+    {
+      "file": "model-00021-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.27.mlp.experts.w13_weight": "1d25b2bdb0e0c98c25a7f14941813dc3b944b4ed543ad20560e01410a0363b00",
+        "model.llm.layers.27.mlp.experts.w2_weight": "2057d9707803a3cb6dd37aeba18644d4bc9e318080868d141f563147f373a1ae"
+      },
+      "seconds": 16.2
+    },
+    {
+      "file": "model-00022-of-00048.safetensors",
+      "bytes": 12445298408,
+      "tensor_bytes": 12445295616,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.28.mlp.experts.w13_weight": "6ae4e54bc677e990864ccd4824a9d9cc9edc19dadde3fa1dbcf7629d37aa1716",
+        "model.llm.layers.28.mlp.experts.w2_weight": "0ba81a8ba62b4b8470bfd0e114c0235a992ef43e81b695c055eb3a3eeb1c58e8"
+      },
+      "seconds": 14.44
+    },
+    {
+      "file": "model-00023-of-00048.safetensors",
+      "bytes": 12444726984,
+      "tensor_bytes": 12444724224,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.29.mlp.experts.w13_weight": "c7078493a18bacd85447514427bb7943bfb5d1dca63a0fb9bc01d76888647c56",
+        "model.llm.layers.29.mlp.experts.w2_weight": "d4aa963f420481313263212434395da62011ac2c3a3b8348e8ae464baee909d1"
+      },
+      "seconds": 14.83
+    },
+    {
+      "file": "model-00024-of-00048.safetensors",
+      "bytes": 12444708336,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.3.mlp.experts.w13_weight": "d32561c49ad404f85625733d255920b6f01f47909df07ae18d3280741c894b85",
+        "model.llm.layers.3.mlp.experts.w2_weight": "889cfb37ec770333c57a3cde3373e4ea47b5f776cf0a597e0fcc78ce0391dc99"
+      },
+      "seconds": 14.84
+    },
+    {
+      "file": "model-00025-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.30.mlp.experts.w13_weight": "24da371081aee501d6a67cc1728f79b2decdcdd2ed5c7f9fdbb35cafec557440",
+        "model.llm.layers.30.mlp.experts.w2_weight": "d1bd7b0afd3d3783e10ae25348cc4693ffcb221506b273db5abaf3a9f789ab1f"
+      },
+      "seconds": 16.13
+    },
+    {
+      "file": "model-00026-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.31.mlp.experts.w13_weight": "ef36bce7785ce47e6933ba3845d7116a06494cd1ef71896110713747bf61edba",
+        "model.llm.layers.31.mlp.experts.w2_weight": "b904bc081a7bd83c1feafd5f10deed768ec728e2b692de03654f6a5b931af482"
+      },
+      "seconds": 15.67
+    },
+    {
+      "file": "model-00027-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.32.mlp.experts.w13_weight": "f40e416bb3503cb712a05e6b4ebbb141be4a06da6f2ada8620400f65f5583ab0",
+        "model.llm.layers.32.mlp.experts.w2_weight": "1536772a2082951ab43cd856b944cacc21d4a8dffef7638541627e4174c3b287"
+      },
+      "seconds": 15.07
+    },
+    {
+      "file": "model-00028-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.33.mlp.experts.w13_weight": "f227658ef4f76b4442b1e84697bc9290a1e4f23137843e5eae129e69edac60ff",
+        "model.llm.layers.33.mlp.experts.w2_weight": "29a67a9b079a8b496e3e5c73c44e517baf354d3985cbba04f8d660cc746589cf"
+      },
+      "seconds": 15.35
+    },
+    {
+      "file": "model-00029-of-00048.safetensors",
+      "bytes": 12445298408,
+      "tensor_bytes": 12445295616,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.34.mlp.experts.w13_weight": "ba265f1bd27ddfc0a8d41c782f98dca00632297540d862f981fc629454dc8cf6",
+        "model.llm.layers.34.mlp.experts.w2_weight": "59cd62ea9869788ffa21d5ee9bb22ca99c20dde2966f3f3276632d02d02b86a7"
+      },
+      "seconds": 16.16
+    },
+    {
+      "file": "model-00030-of-00048.safetensors",
+      "bytes": 12444727000,
+      "tensor_bytes": 12444724224,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.35.mlp.experts.w13_weight": "e31874f8c91f5b704e1667b039d472ebed2b1a8c331a1fe52a5880af8b9a7864",
+        "model.llm.layers.35.mlp.experts.w2_weight": "8d8f91612232bae2550637b3463420409ffcf5a44756281b35edd099094cb2b9"
+      },
+      "seconds": 14.93
+    },
+    {
+      "file": "model-00031-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.36.mlp.experts.w13_weight": "71bce72e5e05bb00d571894602dcd233471538169417fb769374fd10259ef689",
+        "model.llm.layers.36.mlp.experts.w2_weight": "d260af9d6ca264fe8248a9b4ee14cda9c8f1157d845781e0ad22b7b54a8be6c4"
+      },
+      "seconds": 15.37
+    },
+    {
+      "file": "model-00032-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.37.mlp.experts.w13_weight": "4aaa1c1c72f59174fcf5275abd48c64998374bba401c9914e1935e92b9d6f579",
+        "model.llm.layers.37.mlp.experts.w2_weight": "0acfda28033a443dae92915e5def1857eb7ff5cafca4c77243ec2529dc204f4c"
+      },
+      "seconds": 15.52
+    },
+    {
+      "file": "model-00033-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.38.mlp.experts.w13_weight": "cb64404a27d3fedcaf92d993c5bc80af45e31af6878605c02ea15b3d9a735a0f",
+        "model.llm.layers.38.mlp.experts.w2_weight": "17f24d381d1fca08bf503679699bdbc9a522227cd5b8573b0e4ebbfa4f9e47f3"
+      },
+      "seconds": 16.61
+    },
+    {
+      "file": "model-00034-of-00048.safetensors",
+      "bytes": 12444708336,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.39.mlp.experts.w13_weight": "82b92d4bc3e75c92cd5ce8c2ce85d3a0aa3c320f53fc9d87a444c05a20ae3df0",
+        "model.llm.layers.39.mlp.experts.w2_weight": "7b5a6c689a1955d83f3bc49e2362a54ba1dd44619cc8ed82b74a9b533dd176b9"
+      },
+      "seconds": 15.6
+    },
+    {
+      "file": "model-00035-of-00048.safetensors",
+      "bytes": 12444708336,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.4.mlp.experts.w13_weight": "d8c6fb1e196ef27c3f86feb04c9796a8ce5570115f9c3b27fdfc8774f75925db",
+        "model.llm.layers.4.mlp.experts.w2_weight": "e476edfff5a2c75624e60648814ae00c37b79a846c3f46c24d21f8269f585652"
+      },
+      "seconds": 15.08
+    },
+    {
+      "file": "model-00036-of-00048.safetensors",
+      "bytes": 12445298408,
+      "tensor_bytes": 12445295616,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.40.mlp.experts.w13_weight": "e3a0df9702da5f642d607d175e8c43503d5fd7337a1317239d23c02e15dfe0fa",
+        "model.llm.layers.40.mlp.experts.w2_weight": "c796952d739e8ad776e5218658d2c1328cc3e1dc3a784a4aebc88c926df57358"
+      },
+      "seconds": 15.64
+    },
+    {
+      "file": "model-00037-of-00048.safetensors",
+      "bytes": 12444727000,
+      "tensor_bytes": 12444724224,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.41.mlp.experts.w13_weight": "33dd45e0e75c6081aceac2f1ab61fe6460db775b4d946be981364b9ff55a51ca",
+        "model.llm.layers.41.mlp.experts.w2_weight": "5a25f21348fbf9998c2449d40c8655e0d20e695d2c76dfaf27d713576d9d9721"
+      },
+      "seconds": 16.24
+    },
+    {
+      "file": "model-00038-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.42.mlp.experts.w13_weight": "40d8f732a9d435fbc95818b751dbbbdb4c8da623c2f95f600bfd10a6fed369a9",
+        "model.llm.layers.42.mlp.experts.w2_weight": "dc14847f54051788f6977103011dbf7026e485fb3b78d006c93043058d08a941"
+      },
+      "seconds": 15.04
+    },
+    {
+      "file": "model-00039-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.43.mlp.experts.w13_weight": "815428490b2525a31277df4f8041d3d47eed033147ddef40560bae580e5cd764",
+        "model.llm.layers.43.mlp.experts.w2_weight": "dee11b863d9b037d93493961c7ea5fbec9c5eaa539124fcb36a023e2dd25f7c4"
+      },
+      "seconds": 15.4
+    },
+    {
+      "file": "model-00040-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.44.mlp.experts.w13_weight": "e9aa6cab4e000e3b896897ee6f099530938e32d6fcb6c2eec646c27bcbd10c7d",
+        "model.llm.layers.44.mlp.experts.w2_weight": "5e915b546bedf39095c9150628bbb6c0ed6b16302f71608c712c937841a20b5f"
+      },
+      "seconds": 15.93
+    },
+    {
+      "file": "model-00041-of-00048.safetensors",
+      "bytes": 12444708344,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.45.mlp.experts.w13_weight": "faec7dd5d29d4a885e3bcdfe8cc75a82db1a0bd39a76c98b7108a29a073c0d5b",
+        "model.llm.layers.45.mlp.experts.w2_weight": "3a5ba0ec4b54f25b15a5501b68c65a813ac6347783a164d15d2ea999e261524a"
+      },
+      "seconds": 17.01
+    },
+    {
+      "file": "model-00042-of-00048.safetensors",
+      "bytes": 12445298408,
+      "tensor_bytes": 12445295616,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.46.mlp.experts.w13_weight": "8021b5e91f7a89a82b9fb62147f19ff9a105ea0c36aaa879661f615a5e1f198b",
+        "model.llm.layers.46.mlp.experts.w2_weight": "2a069895f88c200c1f2ab20aa620e579d5ff8c1c9fc3adfbec10c4c51d73d9aa"
+      },
+      "seconds": 15.62
+    },
+    {
+      "file": "model-00043-of-00048.safetensors",
+      "bytes": 12445317040,
+      "tensor_bytes": 12445314048,
+      "tensor_count": 26,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.47.mlp.experts.w13_weight": "b1329d7cee24b3d9cc3be103b5beb071ef7be62eb9162af3686637c1565389d6",
+        "model.llm.layers.47.mlp.experts.w2_weight": "6b9543107bc86bd7b147d89030224094a29ad1c9f485e70c4a1f5c7dd6630995"
+      },
+      "seconds": 14.86
+    },
+    {
+      "file": "model-00044-of-00048.safetensors",
+      "bytes": 12444726976,
+      "tensor_bytes": 12444724224,
+      "tensor_count": 24,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.5.mlp.experts.w13_weight": "46c8e834df33663207d62b3e8338b86b5b9fe049fb3d20a75000c86de4db384a",
+        "model.llm.layers.5.mlp.experts.w2_weight": "9d8caaa208e07111ed16a24664b871f446eb10a51e842929e0ef3eaa3a71a971"
+      },
+      "seconds": 15.93
+    },
+    {
+      "file": "model-00045-of-00048.safetensors",
+      "bytes": 12444708328,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.6.mlp.experts.w13_weight": "9e1a0bea190e5fb2bb78eb91e08fc18ce174d7278801c7bf17c8a77cace1ac84",
+        "model.llm.layers.6.mlp.experts.w2_weight": "7b9af24a602f3a9f38aaab9a6c2e8e1eae3f09f7a2c16ff016396adbf87e422d"
+      },
+      "seconds": 16.21
+    },
+    {
+      "file": "model-00046-of-00048.safetensors",
+      "bytes": 12444708328,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.7.mlp.experts.w13_weight": "0d97a25a16d4334816c77ddf1b31573292826ddd6394283118c44fc5ee48c49e",
+        "model.llm.layers.7.mlp.experts.w2_weight": "bc005f4088415aa4f4ede6be71fd56c433ef2dec470cc0d7acd9e4bfa541c775"
+      },
+      "seconds": 15.54
+    },
+    {
+      "file": "model-00047-of-00048.safetensors",
+      "bytes": 12444708328,
+      "tensor_bytes": 12444705792,
+      "tensor_count": 22,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.8.mlp.experts.w13_weight": "68b6f9a1bd2d823398264d8e43c000bf8c530af033862ea6baee48015fb8d649",
+        "model.llm.layers.8.mlp.experts.w2_weight": "e457cfe45794ec67df31648694e153e8eecc8fd2da6859b6fa9879d1201d3376"
+      },
+      "seconds": 15.45
+    },
+    {
+      "file": "model-00048-of-00048.safetensors",
+      "bytes": 14756419216,
+      "tensor_bytes": 14756365824,
+      "tensor_count": 432,
+      "expert_weights": 2,
+      "sample_fingerprints": {
+        "model.llm.layers.9.mlp.experts.w13_weight": "f0ea99ca9ee79bf5e1ab376e0e3ff098d9c035c6c140db90cd43cdd64087f6ac",
+        "model.llm.layers.9.mlp.experts.w2_weight": "9f8165442e52b766a4cf019e80adb6a4e3265b62e846740ebfde0a84b7011d3c"
+      },
+      "seconds": 16.78
+    }
+  ]
+}

hf_quant_config.json

@@ -0,0 +1,1666 @@
+{
+  "model": {
+    "parameter_groups": [
+      {
+        "compute_dtype": "float32",
+        "output_dtype": "float32",
+        "required_activation_quantization": null,
+        "tensors": [
+          "model.llm.attn_res.attn_queries.0",
+          "model.llm.attn_res.attn_queries.1",
+          "model.llm.attn_res.attn_queries.10",
+          "model.llm.attn_res.attn_queries.11",
+          "model.llm.attn_res.attn_queries.12",
+          "model.llm.attn_res.attn_queries.13",
+          "model.llm.attn_res.attn_queries.14",
+          "model.llm.attn_res.attn_queries.15",
+          "model.llm.attn_res.attn_queries.16",
+          "model.llm.attn_res.attn_queries.17",
+          "model.llm.attn_res.attn_queries.18",
+          "model.llm.attn_res.attn_queries.19",
+          "model.llm.attn_res.attn_queries.2",
+          "model.llm.attn_res.attn_queries.20",
+          "model.llm.attn_res.attn_queries.21",
+          "model.llm.attn_res.attn_queries.22",
+          "model.llm.attn_res.attn_queries.23",
+          "model.llm.attn_res.attn_queries.24",
+          "model.llm.attn_res.attn_queries.25",
+          "model.llm.attn_res.attn_queries.26",
+          "model.llm.attn_res.attn_queries.27",
+          "model.llm.attn_res.attn_queries.28",
+          "model.llm.attn_res.attn_queries.29",
+          "model.llm.attn_res.attn_queries.3",
+          "model.llm.attn_res.attn_queries.30",
+          "model.llm.attn_res.attn_queries.31",
+          "model.llm.attn_res.attn_queries.32",
+          "model.llm.attn_res.attn_queries.33",
+          "model.llm.attn_res.attn_queries.34",
+          "model.llm.attn_res.attn_queries.35",
+          "model.llm.attn_res.attn_queries.36",
+          "model.llm.attn_res.attn_queries.37",
+          "model.llm.attn_res.attn_queries.38",
+          "model.llm.attn_res.attn_queries.39",
+          "model.llm.attn_res.attn_queries.4",
+          "model.llm.attn_res.attn_queries.40",
+          "model.llm.attn_res.attn_queries.41",
+          "model.llm.attn_res.attn_queries.42",
+          "model.llm.attn_res.attn_queries.43",
+          "model.llm.attn_res.attn_queries.44",
+          "model.llm.attn_res.attn_queries.45",
+          "model.llm.attn_res.attn_queries.46",
+          "model.llm.attn_res.attn_queries.5",
+          "model.llm.attn_res.attn_queries.6",
+          "model.llm.attn_res.attn_queries.7",
+          "model.llm.attn_res.attn_queries.8",
+          "model.llm.attn_res.attn_queries.9",
+          "model.llm.attn_res.mlp_queries.0",
+          "model.llm.attn_res.mlp_queries.1",
+          "model.llm.attn_res.mlp_queries.10",
+          "model.llm.attn_res.mlp_queries.11",
+          "model.llm.attn_res.mlp_queries.12",
+          "model.llm.attn_res.mlp_queries.13",
+          "model.llm.attn_res.mlp_queries.14",
+          "model.llm.attn_res.mlp_queries.15",
+          "model.llm.attn_res.mlp_queries.16",
+          "model.llm.attn_res.mlp_queries.17",
+          "model.llm.attn_res.mlp_queries.18",
+          "model.llm.attn_res.mlp_queries.19",
+          "model.llm.attn_res.mlp_queries.2",
+          "model.llm.attn_res.mlp_queries.20",
+          "model.llm.attn_res.mlp_queries.21",
+          "model.llm.attn_res.mlp_queries.22",
+          "model.llm.attn_res.mlp_queries.23",
+          "model.llm.attn_res.mlp_queries.24",
+          "model.llm.attn_res.mlp_queries.25",
+          "model.llm.attn_res.mlp_queries.26",
+          "model.llm.attn_res.mlp_queries.27",
+          "model.llm.attn_res.mlp_queries.28",
+          "model.llm.attn_res.mlp_queries.29",
+          "model.llm.attn_res.mlp_queries.3",
+          "model.llm.attn_res.mlp_queries.30",
+          "model.llm.attn_res.mlp_queries.31",
+          "model.llm.attn_res.mlp_queries.32",
+          "model.llm.attn_res.mlp_queries.33",
+          "model.llm.attn_res.mlp_queries.34",
+          "model.llm.attn_res.mlp_queries.35",
+          "model.llm.attn_res.mlp_queries.36",
+          "model.llm.attn_res.mlp_queries.37",
+          "model.llm.attn_res.mlp_queries.38",
+          "model.llm.attn_res.mlp_queries.39",
+          "model.llm.attn_res.mlp_queries.4",
+          "model.llm.attn_res.mlp_queries.40",
+          "model.llm.attn_res.mlp_queries.41",
+          "model.llm.attn_res.mlp_queries.42",
+          "model.llm.attn_res.mlp_queries.43",
+          "model.llm.attn_res.mlp_queries.44",
+          "model.llm.attn_res.mlp_queries.45",
+          "model.llm.attn_res.mlp_queries.46",
+          "model.llm.attn_res.mlp_queries.47",
+          "model.llm.attn_res.mlp_queries.5",
+          "model.llm.attn_res.mlp_queries.6",
+          "model.llm.attn_res.mlp_queries.7",
+          "model.llm.attn_res.mlp_queries.8",
+          "model.llm.attn_res.mlp_queries.9",
+          "model.llm.attn_res.output_query",
+          "model.llm.layers.0.mlp.gate.bias",
+          "model.llm.layers.1.mlp.gate.bias",
+          "model.llm.layers.10.mlp.gate.bias",
+          "model.llm.layers.11.mlp.gate.bias",
+          "model.llm.layers.12.mlp.gate.bias",
+          "model.llm.layers.13.mlp.gate.bias",
+          "model.llm.layers.14.mlp.gate.bias",
+          "model.llm.layers.15.mlp.gate.bias",
+          "model.llm.layers.16.mlp.gate.bias",
+          "model.llm.layers.17.mlp.gate.bias",
+          "model.llm.layers.18.mlp.gate.bias",
+          "model.llm.layers.19.mlp.gate.bias",
+          "model.llm.layers.2.mlp.gate.bias",
+          "model.llm.layers.20.mlp.gate.bias",
+          "model.llm.layers.21.mlp.gate.bias",
+          "model.llm.layers.22.mlp.gate.bias",
+          "model.llm.layers.23.mlp.gate.bias",
+          "model.llm.layers.24.mlp.gate.bias",
+          "model.llm.layers.25.mlp.gate.bias",
+          "model.llm.layers.26.mlp.gate.bias",
+          "model.llm.layers.27.mlp.gate.bias",
+          "model.llm.layers.28.mlp.gate.bias",
+          "model.llm.layers.29.mlp.gate.bias",
+          "model.llm.layers.3.mlp.gate.bias",
+          "model.llm.layers.30.mlp.gate.bias",
+          "model.llm.layers.31.mlp.gate.bias",
+          "model.llm.layers.32.mlp.gate.bias",
+          "model.llm.layers.33.mlp.gate.bias",
+          "model.llm.layers.34.mlp.gate.bias",
+          "model.llm.layers.35.mlp.gate.bias",
+          "model.llm.layers.36.mlp.gate.bias",
+          "model.llm.layers.37.mlp.gate.bias",
+          "model.llm.layers.38.mlp.gate.bias",
+          "model.llm.layers.39.mlp.gate.bias",
+          "model.llm.layers.4.mlp.gate.bias",
+          "model.llm.layers.40.mlp.gate.bias",
+          "model.llm.layers.41.mlp.gate.bias",
+          "model.llm.layers.42.mlp.gate.bias",
+          "model.llm.layers.43.mlp.gate.bias",
+          "model.llm.layers.44.mlp.gate.bias",
+          "model.llm.layers.45.mlp.gate.bias",
+          "model.llm.layers.46.mlp.gate.bias",
+          "model.llm.layers.47.mlp.gate.bias",
+          "model.llm.layers.5.mlp.gate.bias",
+          "model.llm.layers.6.mlp.gate.bias",
+          "model.llm.layers.7.mlp.gate.bias",
+          "model.llm.layers.8.mlp.gate.bias",
+          "model.llm.layers.9.mlp.gate.bias"
+        ],
+        "weight_block_shape": [],
+        "weight_transform": "cast"
+      },
+      {
+        "compute_dtype": "bfloat16",
+        "output_dtype": "bfloat16",
+        "required_activation_quantization": null,
+        "tensors": [
+          "model.llm.embed.weight",
+          "model.llm.embed_norm.weight",
+          "model.llm.layers.0.attn.attn_sink_proj.weight",
+          "model.llm.layers.0.attn.gate_attn.weight",
+          "model.llm.layers.0.attn.k_sconv.weight",
+          "model.llm.layers.0.attn.q_l2_clip.scale",
+          "model.llm.layers.0.attn.rel_l2_clip.scale",
+          "model.llm.layers.0.attn.rel_logits_proj.proj",
+          "model.llm.layers.0.attn.v_sconv.weight",
+          "model.llm.layers.0.attn.wk_dv.weight",
+          "model.llm.layers.0.attn.wo_ud.weight",
+          "model.llm.layers.0.attn.wq_du.weight",
+          "model.llm.layers.0.attn.wr_du.weight",
+          "model.llm.layers.0.attn.wv_dv.weight",
+          "model.llm.layers.0.attn_norm.weight",
+          "model.llm.layers.0.attn_sconv.weight",
+          "model.llm.layers.0.mlp.gate.weight",
+          "model.llm.layers.0.mlp_norm.weight",
+          "model.llm.layers.0.mlp_sconv.weight",
+          "model.llm.layers.1.attn.attn_sink_proj.weight",
+          "model.llm.layers.1.attn.gate_attn.weight",
+          "model.llm.layers.1.attn.k_sconv.weight",
+          "model.llm.layers.1.attn.q_l2_clip.scale",
+          "model.llm.layers.1.attn.rel_l2_clip.scale",
+          "model.llm.layers.1.attn.rel_logits_proj.proj",
+          "model.llm.layers.1.attn.v_sconv.weight",
+          "model.llm.layers.1.attn.wk_dv.weight",
+          "model.llm.layers.1.attn.wo_ud.weight",
+          "model.llm.layers.1.attn.wq_du.weight",
+          "model.llm.layers.1.attn.wr_du.weight",
+          "model.llm.layers.1.attn.wv_dv.weight",
+          "model.llm.layers.1.attn_norm.weight",
+          "model.llm.layers.1.attn_sconv.weight",
+          "model.llm.layers.1.mlp.gate.weight",
+          "model.llm.layers.1.mlp_norm.weight",
+          "model.llm.layers.1.mlp_sconv.weight",
+          "model.llm.layers.10.attn.attn_sink_proj.weight",
+          "model.llm.layers.10.attn.gate_attn.weight",
+          "model.llm.layers.10.attn.k_sconv.weight",
+          "model.llm.layers.10.attn.q_l2_clip.scale",
+          "model.llm.layers.10.attn.rel_l2_clip.scale",
+          "model.llm.layers.10.attn.rel_logits_proj.proj",
+          "model.llm.layers.10.attn.v_sconv.weight",
+          "model.llm.layers.10.attn.wk_dv.weight",
+          "model.llm.layers.10.attn.wo_ud.weight",
+          "model.llm.layers.10.attn.wq_du.weight",
+          "model.llm.layers.10.attn.wr_du.weight",
+          "model.llm.layers.10.attn.wv_dv.weight",
+          "model.llm.layers.10.attn_norm.weight",
+          "model.llm.layers.10.attn_sconv.weight",
+          "model.llm.layers.10.mlp.gate.weight",
+          "model.llm.layers.10.mlp_norm.weight",
+          "model.llm.layers.10.mlp_sconv.weight",
+          "model.llm.layers.11.attn.attn_sink_proj.weight",
+          "model.llm.layers.11.attn.gate_attn.weight",
+          "model.llm.layers.11.attn.k_sconv.static_w",
+          "model.llm.layers.11.attn.k_sconv.weight_proj",
+          "model.llm.layers.11.attn.q_l2_clip.scale",
+          "model.llm.layers.11.attn.rel_l2_clip.scale",
+          "model.llm.layers.11.attn.rel_logits_proj.proj",
+          "model.llm.layers.11.attn.v_sconv.static_w",
+          "model.llm.layers.11.attn.v_sconv.weight_proj",
+          "model.llm.layers.11.attn.wk_dv.weight",
+          "model.llm.layers.11.attn.wo_ud.weight",
+          "model.llm.layers.11.attn.wq_du.weight",
+          "model.llm.layers.11.attn.wr_du.weight",
+          "model.llm.layers.11.attn.wv_dv.weight",
+          "model.llm.layers.11.attn_sconv.weight",
+          "model.llm.layers.11.k_input_norm.weight",
+          "model.llm.layers.11.mlp.gate.weight",
+          "model.llm.layers.11.mlp_norm.weight",
+          "model.llm.layers.11.mlp_sconv.weight",
+          "model.llm.layers.11.q_input_norm.weight",
+          "model.llm.layers.11.v_input_norm.weight",
+          "model.llm.layers.12.attn.attn_sink_proj.weight",
+          "model.llm.layers.12.attn.gate_attn.weight",
+          "model.llm.layers.12.attn.k_sconv.weight",
+          "model.llm.layers.12.attn.q_l2_clip.scale",
+          "model.llm.layers.12.attn.rel_l2_clip.scale",
+          "model.llm.layers.12.attn.rel_logits_proj.proj",
+          "model.llm.layers.12.attn.v_sconv.weight",
+          "model.llm.layers.12.attn.wk_dv.weight",
+          "model.llm.layers.12.attn.wo_ud.weight",
+          "model.llm.layers.12.attn.wq_du.weight",
+          "model.llm.layers.12.attn.wr_du.weight",
+          "model.llm.layers.12.attn.wv_dv.weight",
+          "model.llm.layers.12.attn_norm.weight",
+          "model.llm.layers.12.attn_sconv.weight",
+          "model.llm.layers.12.mlp.gate.weight",
+          "model.llm.layers.12.mlp_norm.weight",
+          "model.llm.layers.12.mlp_sconv.weight",
+          "model.llm.layers.13.attn.attn_sink_proj.weight",
+          "model.llm.layers.13.attn.gate_attn.weight",
+          "model.llm.layers.13.attn.k_sconv.weight",
+          "model.llm.layers.13.attn.q_l2_clip.scale",
+          "model.llm.layers.13.attn.rel_l2_clip.scale",
+          "model.llm.layers.13.attn.rel_logits_proj.proj",
+          "model.llm.layers.13.attn.v_sconv.weight",
+          "model.llm.layers.13.attn.wk_dv.weight",
+          "model.llm.layers.13.attn.wo_ud.weight",
+          "model.llm.layers.13.attn.wq_du.weight",
+          "model.llm.layers.13.attn.wr_du.weight",
+          "model.llm.layers.13.attn.wv_dv.weight",
+          "model.llm.layers.13.attn_norm.weight",
+          "model.llm.layers.13.attn_sconv.weight",
+          "model.llm.layers.13.mlp.gate.weight",
+          "model.llm.layers.13.mlp_norm.weight",
+          "model.llm.layers.13.mlp_sconv.weight",
+          "model.llm.layers.14.attn.attn_sink_proj.weight",
+          "model.llm.layers.14.attn.gate_attn.weight",
+          "model.llm.layers.14.attn.k_sconv.weight",
+          "model.llm.layers.14.attn.q_l2_clip.scale",
+          "model.llm.layers.14.attn.rel_l2_clip.scale",
+          "model.llm.layers.14.attn.rel_logits_proj.proj",
+          "model.llm.layers.14.attn.v_sconv.weight",
+          "model.llm.layers.14.attn.wk_dv.weight",
+          "model.llm.layers.14.attn.wo_ud.weight",
+          "model.llm.layers.14.attn.wq_du.weight",
+          "model.llm.layers.14.attn.wr_du.weight",
+          "model.llm.layers.14.attn.wv_dv.weight",
+          "model.llm.layers.14.attn_norm.weight",
+          "model.llm.layers.14.attn_sconv.weight",
+          "model.llm.layers.14.mlp.gate.weight",
+          "model.llm.layers.14.mlp_norm.weight",
+          "model.llm.layers.14.mlp_sconv.weight",
+          "model.llm.layers.15.attn.attn_sink_proj.weight",
+          "model.llm.layers.15.attn.gate_attn.weight",
+          "model.llm.layers.15.attn.k_sconv.weight",
+          "model.llm.layers.15.attn.q_l2_clip.scale",
+          "model.llm.layers.15.attn.rel_l2_clip.scale",
+          "model.llm.layers.15.attn.rel_logits_proj.proj",
+          "model.llm.layers.15.attn.v_sconv.weight",
+          "model.llm.layers.15.attn.wk_dv.weight",
+          "model.llm.layers.15.attn.wo_ud.weight",
+          "model.llm.layers.15.attn.wq_du.weight",
+          "model.llm.layers.15.attn.wr_du.weight",
+          "model.llm.layers.15.attn.wv_dv.weight",
+          "model.llm.layers.15.attn_norm.weight",
+          "model.llm.layers.15.attn_sconv.weight",
+          "model.llm.layers.15.mlp.gate.weight",
+          "model.llm.layers.15.mlp_norm.weight",
+          "model.llm.layers.15.mlp_sconv.weight",
+          "model.llm.layers.16.attn.attn_sink_proj.weight",
+          "model.llm.layers.16.attn.gate_attn.weight",
+          "model.llm.layers.16.attn.k_sconv.weight",
+          "model.llm.layers.16.attn.q_l2_clip.scale",
+          "model.llm.layers.16.attn.rel_l2_clip.scale",
+          "model.llm.layers.16.attn.rel_logits_proj.proj",
+          "model.llm.layers.16.attn.v_sconv.weight",
+          "model.llm.layers.16.attn.wk_dv.weight",
+          "model.llm.layers.16.attn.wo_ud.weight",
+          "model.llm.layers.16.attn.wq_du.weight",
+          "model.llm.layers.16.attn.wr_du.weight",
+          "model.llm.layers.16.attn.wv_dv.weight",
+          "model.llm.layers.16.attn_norm.weight",
+          "model.llm.layers.16.attn_sconv.weight",
+          "model.llm.layers.16.mlp.gate.weight",
+          "model.llm.layers.16.mlp_norm.weight",
+          "model.llm.layers.16.mlp_sconv.weight",
+          "model.llm.layers.17.attn.attn_sink_proj.weight",
+          "model.llm.layers.17.attn.gate_attn.weight",
+          "model.llm.layers.17.attn.k_sconv.static_w",
+          "model.llm.layers.17.attn.k_sconv.weight_proj",
+          "model.llm.layers.17.attn.q_l2_clip.scale",
+          "model.llm.layers.17.attn.rel_l2_clip.scale",
+          "model.llm.layers.17.attn.rel_logits_proj.proj",
+          "model.llm.layers.17.attn.v_sconv.static_w",
+          "model.llm.layers.17.attn.v_sconv.weight_proj",
+          "model.llm.layers.17.attn.wk_dv.weight",
+          "model.llm.layers.17.attn.wo_ud.weight",
+          "model.llm.layers.17.attn.wq_du.weight",
+          "model.llm.layers.17.attn.wr_du.weight",
+          "model.llm.layers.17.attn.wv_dv.weight",
+          "model.llm.layers.17.attn_sconv.weight",
+          "model.llm.layers.17.k_input_norm.weight",
+          "model.llm.layers.17.mlp.gate.weight",
+          "model.llm.layers.17.mlp_norm.weight",
+          "model.llm.layers.17.mlp_sconv.weight",
+          "model.llm.layers.17.q_input_norm.weight",
+          "model.llm.layers.17.v_input_norm.weight",
+          "model.llm.layers.18.attn.attn_sink_proj.weight",
+          "model.llm.layers.18.attn.gate_attn.weight",
+          "model.llm.layers.18.attn.k_sconv.weight",
+          "model.llm.layers.18.attn.q_l2_clip.scale",
+          "model.llm.layers.18.attn.rel_l2_clip.scale",
+          "model.llm.layers.18.attn.rel_logits_proj.proj",
+          "model.llm.layers.18.attn.v_sconv.weight",
+          "model.llm.layers.18.attn.wk_dv.weight",
+          "model.llm.layers.18.attn.wo_ud.weight",
+          "model.llm.layers.18.attn.wq_du.weight",
+          "model.llm.layers.18.attn.wr_du.weight",
+          "model.llm.layers.18.attn.wv_dv.weight",
+          "model.llm.layers.18.attn_norm.weight",
+          "model.llm.layers.18.attn_sconv.weight",
+          "model.llm.layers.18.mlp.gate.weight",
+          "model.llm.layers.18.mlp_norm.weight",
+          "model.llm.layers.18.mlp_sconv.weight",
+          "model.llm.layers.19.attn.attn_sink_proj.weight",
+          "model.llm.layers.19.attn.gate_attn.weight",
+          "model.llm.layers.19.attn.k_sconv.weight",
+          "model.llm.layers.19.attn.q_l2_clip.scale",
+          "model.llm.layers.19.attn.rel_l2_clip.scale",
+          "model.llm.layers.19.attn.rel_logits_proj.proj",
+          "model.llm.layers.19.attn.v_sconv.weight",
+          "model.llm.layers.19.attn.wk_dv.weight",
+          "model.llm.layers.19.attn.wo_ud.weight",
+          "model.llm.layers.19.attn.wq_du.weight",
+          "model.llm.layers.19.attn.wr_du.weight",
+          "model.llm.layers.19.attn.wv_dv.weight",
+          "model.llm.layers.19.attn_norm.weight",
+          "model.llm.layers.19.attn_sconv.weight",
+          "model.llm.layers.19.mlp.gate.weight",
+          "model.llm.layers.19.mlp_norm.weight",
+          "model.llm.layers.19.mlp_sconv.weight",
+          "model.llm.layers.2.attn.attn_sink_proj.weight",
+          "model.llm.layers.2.attn.gate_attn.weight",
+          "model.llm.layers.2.attn.k_sconv.weight",
+          "model.llm.layers.2.attn.q_l2_clip.scale",
+          "model.llm.layers.2.attn.rel_l2_clip.scale",
+          "model.llm.layers.2.attn.rel_logits_proj.proj",
+          "model.llm.layers.2.attn.v_sconv.weight",
+          "model.llm.layers.2.attn.wk_dv.weight",
+          "model.llm.layers.2.attn.wo_ud.weight",
+          "model.llm.layers.2.attn.wq_du.weight",
+          "model.llm.layers.2.attn.wr_du.weight",
+          "model.llm.layers.2.attn.wv_dv.weight",
+          "model.llm.layers.2.attn_norm.weight",
+          "model.llm.layers.2.attn_sconv.weight",
+          "model.llm.layers.2.mlp.gate.weight",
+          "model.llm.layers.2.mlp_norm.weight",
+          "model.llm.layers.2.mlp_sconv.weight",
+          "model.llm.layers.20.attn.attn_sink_proj.weight",
+          "model.llm.layers.20.attn.gate_attn.weight",
+          "model.llm.layers.20.attn.k_sconv.weight",
+          "model.llm.layers.20.attn.q_l2_clip.scale",
+          "model.llm.layers.20.attn.rel_l2_clip.scale",
+          "model.llm.layers.20.attn.rel_logits_proj.proj",
+          "model.llm.layers.20.attn.v_sconv.weight",
+          "model.llm.layers.20.attn.wk_dv.weight",
+          "model.llm.layers.20.attn.wo_ud.weight",
+          "model.llm.layers.20.attn.wq_du.weight",
+          "model.llm.layers.20.attn.wr_du.weight",
+          "model.llm.layers.20.attn.wv_dv.weight",
+          "model.llm.layers.20.attn_norm.weight",
+          "model.llm.layers.20.attn_sconv.weight",
+          "model.llm.layers.20.mlp.gate.weight",
+          "model.llm.layers.20.mlp_norm.weight",
+          "model.llm.layers.20.mlp_sconv.weight",
+          "model.llm.layers.21.attn.attn_sink_proj.weight",
+          "model.llm.layers.21.attn.gate_attn.weight",
+          "model.llm.layers.21.attn.k_sconv.weight",
+          "model.llm.layers.21.attn.q_l2_clip.scale",
+          "model.llm.layers.21.attn.rel_l2_clip.scale",
+          "model.llm.layers.21.attn.rel_logits_proj.proj",
+          "model.llm.layers.21.attn.v_sconv.weight",
+          "model.llm.layers.21.attn.wk_dv.weight",
+          "model.llm.layers.21.attn.wo_ud.weight",
+          "model.llm.layers.21.attn.wq_du.weight",
+          "model.llm.layers.21.attn.wr_du.weight",
+          "model.llm.layers.21.attn.wv_dv.weight",
+          "model.llm.layers.21.attn_norm.weight",
+          "model.llm.layers.21.attn_sconv.weight",
+          "model.llm.layers.21.mlp.gate.weight",
+          "model.llm.layers.21.mlp_norm.weight",
+          "model.llm.layers.21.mlp_sconv.weight",
+          "model.llm.layers.22.attn.attn_sink_proj.weight",
+          "model.llm.layers.22.attn.gate_attn.weight",
+          "model.llm.layers.22.attn.k_sconv.weight",
+          "model.llm.layers.22.attn.q_l2_clip.scale",
+          "model.llm.layers.22.attn.rel_l2_clip.scale",
+          "model.llm.layers.22.attn.rel_logits_proj.proj",
+          "model.llm.layers.22.attn.v_sconv.weight",
+          "model.llm.layers.22.attn.wk_dv.weight",
+          "model.llm.layers.22.attn.wo_ud.weight",
+          "model.llm.layers.22.attn.wq_du.weight",
+          "model.llm.layers.22.attn.wr_du.weight",
+          "model.llm.layers.22.attn.wv_dv.weight",
+          "model.llm.layers.22.attn_norm.weight",
+          "model.llm.layers.22.attn_sconv.weight",
+          "model.llm.layers.22.mlp.gate.weight",
+          "model.llm.layers.22.mlp_norm.weight",
+          "model.llm.layers.22.mlp_sconv.weight",
+          "model.llm.layers.23.attn.attn_sink_proj.weight",
+          "model.llm.layers.23.attn.gate_attn.weight",
+          "model.llm.layers.23.attn.k_sconv.static_w",
+          "model.llm.layers.23.attn.k_sconv.weight_proj",
+          "model.llm.layers.23.attn.q_l2_clip.scale",
+          "model.llm.layers.23.attn.rel_l2_clip.scale",
+          "model.llm.layers.23.attn.rel_logits_proj.proj",
+          "model.llm.layers.23.attn.v_sconv.static_w",
+          "model.llm.layers.23.attn.v_sconv.weight_proj",
+          "model.llm.layers.23.attn.wk_dv.weight",
+          "model.llm.layers.23.attn.wo_ud.weight",
+          "model.llm.layers.23.attn.wq_du.weight",
+          "model.llm.layers.23.attn.wr_du.weight",
+          "model.llm.layers.23.attn.wv_dv.weight",
+          "model.llm.layers.23.attn_sconv.weight",
+          "model.llm.layers.23.k_input_norm.weight",
+          "model.llm.layers.23.mlp.gate.weight",
+          "model.llm.layers.23.mlp_norm.weight",
+          "model.llm.layers.23.mlp_sconv.weight",
+          "model.llm.layers.23.q_input_norm.weight",
+          "model.llm.layers.23.v_input_norm.weight",
+          "model.llm.layers.24.attn.attn_sink_proj.weight",
+          "model.llm.layers.24.attn.gate_attn.weight",
+          "model.llm.layers.24.attn.k_sconv.weight",
+          "model.llm.layers.24.attn.q_l2_clip.scale",
+          "model.llm.layers.24.attn.rel_l2_clip.scale",
+          "model.llm.layers.24.attn.rel_logits_proj.proj",
+          "model.llm.layers.24.attn.v_sconv.weight",
+          "model.llm.layers.24.attn.wk_dv.weight",
+          "model.llm.layers.24.attn.wo_ud.weight",
+          "model.llm.layers.24.attn.wq_du.weight",
+          "model.llm.layers.24.attn.wr_du.weight",
+          "model.llm.layers.24.attn.wv_dv.weight",
+          "model.llm.layers.24.attn_norm.weight",
+          "model.llm.layers.24.attn_sconv.weight",
+          "model.llm.layers.24.mlp.gate.weight",
+          "model.llm.layers.24.mlp_norm.weight",
+          "model.llm.layers.24.mlp_sconv.weight",
+          "model.llm.layers.25.attn.attn_sink_proj.weight",
+          "model.llm.layers.25.attn.gate_attn.weight",
+          "model.llm.layers.25.attn.k_sconv.weight",
+          "model.llm.layers.25.attn.q_l2_clip.scale",
+          "model.llm.layers.25.attn.rel_l2_clip.scale",
+          "model.llm.layers.25.attn.rel_logits_proj.proj",
+          "model.llm.layers.25.attn.v_sconv.weight",
+          "model.llm.layers.25.attn.wk_dv.weight",
+          "model.llm.layers.25.attn.wo_ud.weight",
+          "model.llm.layers.25.attn.wq_du.weight",
+          "model.llm.layers.25.attn.wr_du.weight",
+          "model.llm.layers.25.attn.wv_dv.weight",
+          "model.llm.layers.25.attn_norm.weight",
+          "model.llm.layers.25.attn_sconv.weight",
+          "model.llm.layers.25.mlp.gate.weight",
+          "model.llm.layers.25.mlp_norm.weight",
+          "model.llm.layers.25.mlp_sconv.weight",
+          "model.llm.layers.26.attn.attn_sink_proj.weight",
+          "model.llm.layers.26.attn.gate_attn.weight",
+          "model.llm.layers.26.attn.k_sconv.weight",
+          "model.llm.layers.26.attn.q_l2_clip.scale",
+          "model.llm.layers.26.attn.rel_l2_clip.scale",
+          "model.llm.layers.26.attn.rel_logits_proj.proj",
+          "model.llm.layers.26.attn.v_sconv.weight",
+          "model.llm.layers.26.attn.wk_dv.weight",
+          "model.llm.layers.26.attn.wo_ud.weight",
+          "model.llm.layers.26.attn.wq_du.weight",
+          "model.llm.layers.26.attn.wr_du.weight",
+          "model.llm.layers.26.attn.wv_dv.weight",
+          "model.llm.layers.26.attn_norm.weight",
+          "model.llm.layers.26.attn_sconv.weight",
+          "model.llm.layers.26.mlp.gate.weight",
+          "model.llm.layers.26.mlp_norm.weight",
+          "model.llm.layers.26.mlp_sconv.weight",
+          "model.llm.layers.27.attn.attn_sink_proj.weight",
+          "model.llm.layers.27.attn.gate_attn.weight",
+          "model.llm.layers.27.attn.k_sconv.weight",
+          "model.llm.layers.27.attn.q_l2_clip.scale",
+          "model.llm.layers.27.attn.rel_l2_clip.scale",
+          "model.llm.layers.27.attn.rel_logits_proj.proj",
+          "model.llm.layers.27.attn.v_sconv.weight",
+          "model.llm.layers.27.attn.wk_dv.weight",
+          "model.llm.layers.27.attn.wo_ud.weight",
+          "model.llm.layers.27.attn.wq_du.weight",
+          "model.llm.layers.27.attn.wr_du.weight",
+          "model.llm.layers.27.attn.wv_dv.weight",
+          "model.llm.layers.27.attn_norm.weight",
+          "model.llm.layers.27.attn_sconv.weight",
+          "model.llm.layers.27.mlp.gate.weight",
+          "model.llm.layers.27.mlp_norm.weight",
+          "model.llm.layers.27.mlp_sconv.weight",
+          "model.llm.layers.28.attn.attn_sink_proj.weight",
+          "model.llm.layers.28.attn.gate_attn.weight",
+          "model.llm.layers.28.attn.k_sconv.weight",
+          "model.llm.layers.28.attn.q_l2_clip.scale",
+          "model.llm.layers.28.attn.rel_l2_clip.scale",
+          "model.llm.layers.28.attn.rel_logits_proj.proj",
+          "model.llm.layers.28.attn.v_sconv.weight",
+          "model.llm.layers.28.attn.wk_dv.weight",
+          "model.llm.layers.28.attn.wo_ud.weight",
+          "model.llm.layers.28.attn.wq_du.weight",
+          "model.llm.layers.28.attn.wr_du.weight",
+          "model.llm.layers.28.attn.wv_dv.weight",
+          "model.llm.layers.28.attn_norm.weight",
+          "model.llm.layers.28.attn_sconv.weight",
+          "model.llm.layers.28.mlp.gate.weight",
+          "model.llm.layers.28.mlp_norm.weight",
+          "model.llm.layers.28.mlp_sconv.weight",
+          "model.llm.layers.29.attn.attn_sink_proj.weight",
+          "model.llm.layers.29.attn.gate_attn.weight",
+          "model.llm.layers.29.attn.k_sconv.static_w",
+          "model.llm.layers.29.attn.k_sconv.weight_proj",
+          "model.llm.layers.29.attn.q_l2_clip.scale",
+          "model.llm.layers.29.attn.rel_l2_clip.scale",
+          "model.llm.layers.29.attn.rel_logits_proj.proj",
+          "model.llm.layers.29.attn.v_sconv.static_w",
+          "model.llm.layers.29.attn.v_sconv.weight_proj",
+          "model.llm.layers.29.attn.wk_dv.weight",
+          "model.llm.layers.29.attn.wo_ud.weight",
+          "model.llm.layers.29.attn.wq_du.weight",
+          "model.llm.layers.29.attn.wr_du.weight",
+          "model.llm.layers.29.attn.wv_dv.weight",
+          "model.llm.layers.29.attn_sconv.weight",
+          "model.llm.layers.29.k_input_norm.weight",
+          "model.llm.layers.29.mlp.gate.weight",
+          "model.llm.layers.29.mlp_norm.weight",
+          "model.llm.layers.29.mlp_sconv.weight",
+          "model.llm.layers.29.q_input_norm.weight",
+          "model.llm.layers.29.v_input_norm.weight",
+          "model.llm.layers.3.attn.attn_sink_proj.weight",
+          "model.llm.layers.3.attn.gate_attn.weight",
+          "model.llm.layers.3.attn.k_sconv.weight",
+          "model.llm.layers.3.attn.q_l2_clip.scale",
+          "model.llm.layers.3.attn.rel_l2_clip.scale",
+          "model.llm.layers.3.attn.rel_logits_proj.proj",
+          "model.llm.layers.3.attn.v_sconv.weight",
+          "model.llm.layers.3.attn.wk_dv.weight",
+          "model.llm.layers.3.attn.wo_ud.weight",
+          "model.llm.layers.3.attn.wq_du.weight",
+          "model.llm.layers.3.attn.wr_du.weight",
+          "model.llm.layers.3.attn.wv_dv.weight",
+          "model.llm.layers.3.attn_norm.weight",
+          "model.llm.layers.3.attn_sconv.weight",
+          "model.llm.layers.3.mlp.gate.weight",
+          "model.llm.layers.3.mlp_norm.weight",
+          "model.llm.layers.3.mlp_sconv.weight",
+          "model.llm.layers.30.attn.attn_sink_proj.weight",
+          "model.llm.layers.30.attn.gate_attn.weight",
+          "model.llm.layers.30.attn.k_sconv.weight",
+          "model.llm.layers.30.attn.q_l2_clip.scale",
+          "model.llm.layers.30.attn.rel_l2_clip.scale",
+          "model.llm.layers.30.attn.rel_logits_proj.proj",
+          "model.llm.layers.30.attn.v_sconv.weight",
+          "model.llm.layers.30.attn.wk_dv.weight",
+          "model.llm.layers.30.attn.wo_ud.weight",
+          "model.llm.layers.30.attn.wq_du.weight",
+          "model.llm.layers.30.attn.wr_du.weight",
+          "model.llm.layers.30.attn.wv_dv.weight",
+          "model.llm.layers.30.attn_norm.weight",
+          "model.llm.layers.30.attn_sconv.weight",
+          "model.llm.layers.30.mlp.gate.weight",
+          "model.llm.layers.30.mlp_norm.weight",
+          "model.llm.layers.30.mlp_sconv.weight",
+          "model.llm.layers.31.attn.attn_sink_proj.weight",
+          "model.llm.layers.31.attn.gate_attn.weight",
+          "model.llm.layers.31.attn.k_sconv.weight",
+          "model.llm.layers.31.attn.q_l2_clip.scale",
+          "model.llm.layers.31.attn.rel_l2_clip.scale",
+          "model.llm.layers.31.attn.rel_logits_proj.proj",
+          "model.llm.layers.31.attn.v_sconv.weight",
+          "model.llm.layers.31.attn.wk_dv.weight",
+          "model.llm.layers.31.attn.wo_ud.weight",
+          "model.llm.layers.31.attn.wq_du.weight",
+          "model.llm.layers.31.attn.wr_du.weight",
+          "model.llm.layers.31.attn.wv_dv.weight",
+          "model.llm.layers.31.attn_norm.weight",
+          "model.llm.layers.31.attn_sconv.weight",
+          "model.llm.layers.31.mlp.gate.weight",
+          "model.llm.layers.31.mlp_norm.weight",
+          "model.llm.layers.31.mlp_sconv.weight",
+          "model.llm.layers.32.attn.attn_sink_proj.weight",
+          "model.llm.layers.32.attn.gate_attn.weight",
+          "model.llm.layers.32.attn.k_sconv.weight",
+          "model.llm.layers.32.attn.q_l2_clip.scale",
+          "model.llm.layers.32.attn.rel_l2_clip.scale",
+          "model.llm.layers.32.attn.rel_logits_proj.proj",
+          "model.llm.layers.32.attn.v_sconv.weight",
+          "model.llm.layers.32.attn.wk_dv.weight",
+          "model.llm.layers.32.attn.wo_ud.weight",
+          "model.llm.layers.32.attn.wq_du.weight",
+          "model.llm.layers.32.attn.wr_du.weight",
+          "model.llm.layers.32.attn.wv_dv.weight",
+          "model.llm.layers.32.attn_norm.weight",
+          "model.llm.layers.32.attn_sconv.weight",
+          "model.llm.layers.32.mlp.gate.weight",
+          "model.llm.layers.32.mlp_norm.weight",
+          "model.llm.layers.32.mlp_sconv.weight",
+          "model.llm.layers.33.attn.attn_sink_proj.weight",
+          "model.llm.layers.33.attn.gate_attn.weight",
+          "model.llm.layers.33.attn.k_sconv.weight",
+          "model.llm.layers.33.attn.q_l2_clip.scale",
+          "model.llm.layers.33.attn.rel_l2_clip.scale",
+          "model.llm.layers.33.attn.rel_logits_proj.proj",
+          "model.llm.layers.33.attn.v_sconv.weight",
+          "model.llm.layers.33.attn.wk_dv.weight",
+          "model.llm.layers.33.attn.wo_ud.weight",
+          "model.llm.layers.33.attn.wq_du.weight",
+          "model.llm.layers.33.attn.wr_du.weight",
+          "model.llm.layers.33.attn.wv_dv.weight",
+          "model.llm.layers.33.attn_norm.weight",
+          "model.llm.layers.33.attn_sconv.weight",
+          "model.llm.layers.33.mlp.gate.weight",
+          "model.llm.layers.33.mlp_norm.weight",
+          "model.llm.layers.33.mlp_sconv.weight",
+          "model.llm.layers.34.attn.attn_sink_proj.weight",
+          "model.llm.layers.34.attn.gate_attn.weight",
+          "model.llm.layers.34.attn.k_sconv.weight",
+          "model.llm.layers.34.attn.q_l2_clip.scale",
+          "model.llm.layers.34.attn.rel_l2_clip.scale",
+          "model.llm.layers.34.attn.rel_logits_proj.proj",
+          "model.llm.layers.34.attn.v_sconv.weight",
+          "model.llm.layers.34.attn.wk_dv.weight",
+          "model.llm.layers.34.attn.wo_ud.weight",
+          "model.llm.layers.34.attn.wq_du.weight",
+          "model.llm.layers.34.attn.wr_du.weight",
+          "model.llm.layers.34.attn.wv_dv.weight",
+          "model.llm.layers.34.attn_norm.weight",
+          "model.llm.layers.34.attn_sconv.weight",
+          "model.llm.layers.34.mlp.gate.weight",
+          "model.llm.layers.34.mlp_norm.weight",
+          "model.llm.layers.34.mlp_sconv.weight",
+          "model.llm.layers.35.attn.attn_sink_proj.weight",
+          "model.llm.layers.35.attn.gate_attn.weight",
+          "model.llm.layers.35.attn.k_sconv.static_w",
+          "model.llm.layers.35.attn.k_sconv.weight_proj",
+          "model.llm.layers.35.attn.q_l2_clip.scale",
+          "model.llm.layers.35.attn.rel_l2_clip.scale",
+          "model.llm.layers.35.attn.rel_logits_proj.proj",
+          "model.llm.layers.35.attn.v_sconv.static_w",
+          "model.llm.layers.35.attn.v_sconv.weight_proj",
+          "model.llm.layers.35.attn.wk_dv.weight",
+          "model.llm.layers.35.attn.wo_ud.weight",
+          "model.llm.layers.35.attn.wq_du.weight",
+          "model.llm.layers.35.attn.wr_du.weight",
+          "model.llm.layers.35.attn.wv_dv.weight",
+          "model.llm.layers.35.attn_sconv.weight",
+          "model.llm.layers.35.k_input_norm.weight",
+          "model.llm.layers.35.mlp.gate.weight",
+          "model.llm.layers.35.mlp_norm.weight",
+          "model.llm.layers.35.mlp_sconv.weight",
+          "model.llm.layers.35.q_input_norm.weight",
+          "model.llm.layers.35.v_input_norm.weight",
+          "model.llm.layers.36.attn.attn_sink_proj.weight",
+          "model.llm.layers.36.attn.gate_attn.weight",
+          "model.llm.layers.36.attn.k_sconv.weight",
+          "model.llm.layers.36.attn.q_l2_clip.scale",
+          "model.llm.layers.36.attn.rel_l2_clip.scale",
+          "model.llm.layers.36.attn.rel_logits_proj.proj",
+          "model.llm.layers.36.attn.v_sconv.weight",
+          "model.llm.layers.36.attn.wk_dv.weight",
+          "model.llm.layers.36.attn.wo_ud.weight",
+          "model.llm.layers.36.attn.wq_du.weight",
+          "model.llm.layers.36.attn.wr_du.weight",
+          "model.llm.layers.36.attn.wv_dv.weight",
+          "model.llm.layers.36.attn_norm.weight",
+          "model.llm.layers.36.attn_sconv.weight",
+          "model.llm.layers.36.mlp.gate.weight",
+          "model.llm.layers.36.mlp_norm.weight",
+          "model.llm.layers.36.mlp_sconv.weight",
+          "model.llm.layers.37.attn.attn_sink_proj.weight",
+          "model.llm.layers.37.attn.gate_attn.weight",
+          "model.llm.layers.37.attn.k_sconv.weight",
+          "model.llm.layers.37.attn.q_l2_clip.scale",
+          "model.llm.layers.37.attn.rel_l2_clip.scale",
+          "model.llm.layers.37.attn.rel_logits_proj.proj",
+          "model.llm.layers.37.attn.v_sconv.weight",
+          "model.llm.layers.37.attn.wk_dv.weight",
+          "model.llm.layers.37.attn.wo_ud.weight",
+          "model.llm.layers.37.attn.wq_du.weight",
+          "model.llm.layers.37.attn.wr_du.weight",
+          "model.llm.layers.37.attn.wv_dv.weight",
+          "model.llm.layers.37.attn_norm.weight",
+          "model.llm.layers.37.attn_sconv.weight",
+          "model.llm.layers.37.mlp.gate.weight",
+          "model.llm.layers.37.mlp_norm.weight",
+          "model.llm.layers.37.mlp_sconv.weight",
+          "model.llm.layers.38.attn.attn_sink_proj.weight",
+          "model.llm.layers.38.attn.gate_attn.weight",
+          "model.llm.layers.38.attn.k_sconv.weight",
+          "model.llm.layers.38.attn.q_l2_clip.scale",
+          "model.llm.layers.38.attn.rel_l2_clip.scale",
+          "model.llm.layers.38.attn.rel_logits_proj.proj",
+          "model.llm.layers.38.attn.v_sconv.weight",
+          "model.llm.layers.38.attn.wk_dv.weight",
+          "model.llm.layers.38.attn.wo_ud.weight",
+          "model.llm.layers.38.attn.wq_du.weight",
+          "model.llm.layers.38.attn.wr_du.weight",
+          "model.llm.layers.38.attn.wv_dv.weight",
+          "model.llm.layers.38.attn_norm.weight",
+          "model.llm.layers.38.attn_sconv.weight",
+          "model.llm.layers.38.mlp.gate.weight",
+          "model.llm.layers.38.mlp_norm.weight",
+          "model.llm.layers.38.mlp_sconv.weight",
+          "model.llm.layers.39.attn.attn_sink_proj.weight",
+          "model.llm.layers.39.attn.gate_attn.weight",
+          "model.llm.layers.39.attn.k_sconv.weight",
+          "model.llm.layers.39.attn.q_l2_clip.scale",
+          "model.llm.layers.39.attn.rel_l2_clip.scale",
+          "model.llm.layers.39.attn.rel_logits_proj.proj",
+          "model.llm.layers.39.attn.v_sconv.weight",
+          "model.llm.layers.39.attn.wk_dv.weight",
+          "model.llm.layers.39.attn.wo_ud.weight",
+          "model.llm.layers.39.attn.wq_du.weight",
+          "model.llm.layers.39.attn.wr_du.weight",
+          "model.llm.layers.39.attn.wv_dv.weight",
+          "model.llm.layers.39.attn_norm.weight",
+          "model.llm.layers.39.attn_sconv.weight",
+          "model.llm.layers.39.mlp.gate.weight",
+          "model.llm.layers.39.mlp_norm.weight",
+          "model.llm.layers.39.mlp_sconv.weight",
+          "model.llm.layers.4.attn.attn_sink_proj.weight",
+          "model.llm.layers.4.attn.gate_attn.weight",
+          "model.llm.layers.4.attn.k_sconv.weight",
+          "model.llm.layers.4.attn.q_l2_clip.scale",
+          "model.llm.layers.4.attn.rel_l2_clip.scale",
+          "model.llm.layers.4.attn.rel_logits_proj.proj",
+          "model.llm.layers.4.attn.v_sconv.weight",
+          "model.llm.layers.4.attn.wk_dv.weight",
+          "model.llm.layers.4.attn.wo_ud.weight",
+          "model.llm.layers.4.attn.wq_du.weight",
+          "model.llm.layers.4.attn.wr_du.weight",
+          "model.llm.layers.4.attn.wv_dv.weight",
+          "model.llm.layers.4.attn_norm.weight",
+          "model.llm.layers.4.attn_sconv.weight",
+          "model.llm.layers.4.mlp.gate.weight",
+          "model.llm.layers.4.mlp_norm.weight",
+          "model.llm.layers.4.mlp_sconv.weight",
+          "model.llm.layers.40.attn.attn_sink_proj.weight",
+          "model.llm.layers.40.attn.gate_attn.weight",
+          "model.llm.layers.40.attn.k_sconv.weight",
+          "model.llm.layers.40.attn.q_l2_clip.scale",
+          "model.llm.layers.40.attn.rel_l2_clip.scale",
+          "model.llm.layers.40.attn.rel_logits_proj.proj",
+          "model.llm.layers.40.attn.v_sconv.weight",
+          "model.llm.layers.40.attn.wk_dv.weight",
+          "model.llm.layers.40.attn.wo_ud.weight",
+          "model.llm.layers.40.attn.wq_du.weight",
+          "model.llm.layers.40.attn.wr_du.weight",
+          "model.llm.layers.40.attn.wv_dv.weight",
+          "model.llm.layers.40.attn_norm.weight",
+          "model.llm.layers.40.attn_sconv.weight",
+          "model.llm.layers.40.mlp.gate.weight",
+          "model.llm.layers.40.mlp_norm.weight",
+          "model.llm.layers.40.mlp_sconv.weight",
+          "model.llm.layers.41.attn.attn_sink_proj.weight",
+          "model.llm.layers.41.attn.gate_attn.weight",
+          "model.llm.layers.41.attn.k_sconv.static_w",
+          "model.llm.layers.41.attn.k_sconv.weight_proj",
+          "model.llm.layers.41.attn.q_l2_clip.scale",
+          "model.llm.layers.41.attn.rel_l2_clip.scale",
+          "model.llm.layers.41.attn.rel_logits_proj.proj",
+          "model.llm.layers.41.attn.v_sconv.static_w",
+          "model.llm.layers.41.attn.v_sconv.weight_proj",
+          "model.llm.layers.41.attn.wk_dv.weight",
+          "model.llm.layers.41.attn.wo_ud.weight",
+          "model.llm.layers.41.attn.wq_du.weight",
+          "model.llm.layers.41.attn.wr_du.weight",
+          "model.llm.layers.41.attn.wv_dv.weight",
+          "model.llm.layers.41.attn_sconv.weight",
+          "model.llm.layers.41.k_input_norm.weight",
+          "model.llm.layers.41.mlp.gate.weight",
+          "model.llm.layers.41.mlp_norm.weight",
+          "model.llm.layers.41.mlp_sconv.weight",
+          "model.llm.layers.41.q_input_norm.weight",
+          "model.llm.layers.41.v_input_norm.weight",
+          "model.llm.layers.42.attn.attn_sink_proj.weight",
+          "model.llm.layers.42.attn.gate_attn.weight",
+          "model.llm.layers.42.attn.k_sconv.weight",
+          "model.llm.layers.42.attn.q_l2_clip.scale",
+          "model.llm.layers.42.attn.rel_l2_clip.scale",
+          "model.llm.layers.42.attn.rel_logits_proj.proj",
+          "model.llm.layers.42.attn.v_sconv.weight",
+          "model.llm.layers.42.attn.wk_dv.weight",
+          "model.llm.layers.42.attn.wo_ud.weight",
+          "model.llm.layers.42.attn.wq_du.weight",
+          "model.llm.layers.42.attn.wr_du.weight",
+          "model.llm.layers.42.attn.wv_dv.weight",
+          "model.llm.layers.42.attn_norm.weight",
+          "model.llm.layers.42.attn_sconv.weight",
+          "model.llm.layers.42.mlp.gate.weight",
+          "model.llm.layers.42.mlp_norm.weight",
+          "model.llm.layers.42.mlp_sconv.weight",
+          "model.llm.layers.43.attn.attn_sink_proj.weight",
+          "model.llm.layers.43.attn.gate_attn.weight",
+          "model.llm.layers.43.attn.k_sconv.weight",
+          "model.llm.layers.43.attn.q_l2_clip.scale",
+          "model.llm.layers.43.attn.rel_l2_clip.scale",
+          "model.llm.layers.43.attn.rel_logits_proj.proj",
+          "model.llm.layers.43.attn.v_sconv.weight",
+          "model.llm.layers.43.attn.wk_dv.weight",
+          "model.llm.layers.43.attn.wo_ud.weight",
+          "model.llm.layers.43.attn.wq_du.weight",
+          "model.llm.layers.43.attn.wr_du.weight",
+          "model.llm.layers.43.attn.wv_dv.weight",
+          "model.llm.layers.43.attn_norm.weight",
+          "model.llm.layers.43.attn_sconv.weight",
+          "model.llm.layers.43.mlp.gate.weight",
+          "model.llm.layers.43.mlp_norm.weight",
+          "model.llm.layers.43.mlp_sconv.weight",
+          "model.llm.layers.44.attn.attn_sink_proj.weight",
+          "model.llm.layers.44.attn.gate_attn.weight",
+          "model.llm.layers.44.attn.k_sconv.weight",
+          "model.llm.layers.44.attn.q_l2_clip.scale",
+          "model.llm.layers.44.attn.rel_l2_clip.scale",
+          "model.llm.layers.44.attn.rel_logits_proj.proj",
+          "model.llm.layers.44.attn.v_sconv.weight",
+          "model.llm.layers.44.attn.wk_dv.weight",
+          "model.llm.layers.44.attn.wo_ud.weight",
+          "model.llm.layers.44.attn.wq_du.weight",
+          "model.llm.layers.44.attn.wr_du.weight",
+          "model.llm.layers.44.attn.wv_dv.weight",
+          "model.llm.layers.44.attn_norm.weight",
+          "model.llm.layers.44.attn_sconv.weight",
+          "model.llm.layers.44.mlp.gate.weight",
+          "model.llm.layers.44.mlp_norm.weight",
+          "model.llm.layers.44.mlp_sconv.weight",
+          "model.llm.layers.45.attn.attn_sink_proj.weight",
+          "model.llm.layers.45.attn.gate_attn.weight",
+          "model.llm.layers.45.attn.k_sconv.weight",
+          "model.llm.layers.45.attn.q_l2_clip.scale",
+          "model.llm.layers.45.attn.rel_l2_clip.scale",
+          "model.llm.layers.45.attn.rel_logits_proj.proj",
+          "model.llm.layers.45.attn.v_sconv.weight",
+          "model.llm.layers.45.attn.wk_dv.weight",
+          "model.llm.layers.45.attn.wo_ud.weight",
+          "model.llm.layers.45.attn.wq_du.weight",
+          "model.llm.layers.45.attn.wr_du.weight",
+          "model.llm.layers.45.attn.wv_dv.weight",
+          "model.llm.layers.45.attn_norm.weight",
+          "model.llm.layers.45.attn_sconv.weight",
+          "model.llm.layers.45.mlp.gate.weight",
+          "model.llm.layers.45.mlp_norm.weight",
+          "model.llm.layers.45.mlp_sconv.weight",
+          "model.llm.layers.46.attn.attn_sink_proj.weight",
+          "model.llm.layers.46.attn.gate_attn.weight",
+          "model.llm.layers.46.attn.k_sconv.weight",
+          "model.llm.layers.46.attn.q_l2_clip.scale",
+          "model.llm.layers.46.attn.rel_l2_clip.scale",
+          "model.llm.layers.46.attn.rel_logits_proj.proj",
+          "model.llm.layers.46.attn.v_sconv.weight",
+          "model.llm.layers.46.attn.wk_dv.weight",
+          "model.llm.layers.46.attn.wo_ud.weight",
+          "model.llm.layers.46.attn.wq_du.weight",
+          "model.llm.layers.46.attn.wr_du.weight",
+          "model.llm.layers.46.attn.wv_dv.weight",
+          "model.llm.layers.46.attn_norm.weight",
+          "model.llm.layers.46.attn_sconv.weight",
+          "model.llm.layers.46.mlp.gate.weight",
+          "model.llm.layers.46.mlp_norm.weight",
+          "model.llm.layers.46.mlp_sconv.weight",
+          "model.llm.layers.47.attn.attn_sink_proj.weight",
+          "model.llm.layers.47.attn.gate_attn.weight",
+          "model.llm.layers.47.attn.k_sconv.static_w",
+          "model.llm.layers.47.attn.k_sconv.weight_proj",
+          "model.llm.layers.47.attn.q_l2_clip.scale",
+          "model.llm.layers.47.attn.rel_l2_clip.scale",
+          "model.llm.layers.47.attn.rel_logits_proj.proj",
+          "model.llm.layers.47.attn.v_sconv.static_w",
+          "model.llm.layers.47.attn.v_sconv.weight_proj",
+          "model.llm.layers.47.attn.wk_dv.weight",
+          "model.llm.layers.47.attn.wo_ud.weight",
+          "model.llm.layers.47.attn.wq_du.weight",
+          "model.llm.layers.47.attn.wr_du.weight",
+          "model.llm.layers.47.attn.wv_dv.weight",
+          "model.llm.layers.47.attn_sconv.weight",
+          "model.llm.layers.47.k_input_norm.weight",
+          "model.llm.layers.47.mlp.gate.weight",
+          "model.llm.layers.47.mlp_norm.weight",
+          "model.llm.layers.47.mlp_sconv.weight",
+          "model.llm.layers.47.q_input_norm.weight",
+          "model.llm.layers.47.v_input_norm.weight",
+          "model.llm.layers.5.attn.attn_sink_proj.weight",
+          "model.llm.layers.5.attn.gate_attn.weight",
+          "model.llm.layers.5.attn.k_sconv.static_w",
+          "model.llm.layers.5.attn.k_sconv.weight_proj",
+          "model.llm.layers.5.attn.q_l2_clip.scale",
+          "model.llm.layers.5.attn.rel_l2_clip.scale",
+          "model.llm.layers.5.attn.rel_logits_proj.proj",
+          "model.llm.layers.5.attn.v_sconv.static_w",
+          "model.llm.layers.5.attn.v_sconv.weight_proj",
+          "model.llm.layers.5.attn.wk_dv.weight",
+          "model.llm.layers.5.attn.wo_ud.weight",
+          "model.llm.layers.5.attn.wq_du.weight",
+          "model.llm.layers.5.attn.wr_du.weight",
+          "model.llm.layers.5.attn.wv_dv.weight",
+          "model.llm.layers.5.attn_sconv.weight",
+          "model.llm.layers.5.k_input_norm.weight",
+          "model.llm.layers.5.mlp.gate.weight",
+          "model.llm.layers.5.mlp_norm.weight",
+          "model.llm.layers.5.mlp_sconv.weight",
+          "model.llm.layers.5.q_input_norm.weight",
+          "model.llm.layers.5.v_input_norm.weight",
+          "model.llm.layers.6.attn.attn_sink_proj.weight",
+          "model.llm.layers.6.attn.gate_attn.weight",
+          "model.llm.layers.6.attn.k_sconv.weight",
+          "model.llm.layers.6.attn.q_l2_clip.scale",
+          "model.llm.layers.6.attn.rel_l2_clip.scale",
+          "model.llm.layers.6.attn.rel_logits_proj.proj",
+          "model.llm.layers.6.attn.v_sconv.weight",
+          "model.llm.layers.6.attn.wk_dv.weight",
+          "model.llm.layers.6.attn.wo_ud.weight",
+          "model.llm.layers.6.attn.wq_du.weight",
+          "model.llm.layers.6.attn.wr_du.weight",
+          "model.llm.layers.6.attn.wv_dv.weight",
+          "model.llm.layers.6.attn_norm.weight",
+          "model.llm.layers.6.attn_sconv.weight",
+          "model.llm.layers.6.mlp.gate.weight",
+          "model.llm.layers.6.mlp_norm.weight",
+          "model.llm.layers.6.mlp_sconv.weight",
+          "model.llm.layers.7.attn.attn_sink_proj.weight",
+          "model.llm.layers.7.attn.gate_attn.weight",
+          "model.llm.layers.7.attn.k_sconv.weight",
+          "model.llm.layers.7.attn.q_l2_clip.scale",
+          "model.llm.layers.7.attn.rel_l2_clip.scale",
+          "model.llm.layers.7.attn.rel_logits_proj.proj",
+          "model.llm.layers.7.attn.v_sconv.weight",
+          "model.llm.layers.7.attn.wk_dv.weight",
+          "model.llm.layers.7.attn.wo_ud.weight",
+          "model.llm.layers.7.attn.wq_du.weight",
+          "model.llm.layers.7.attn.wr_du.weight",
+          "model.llm.layers.7.attn.wv_dv.weight",
+          "model.llm.layers.7.attn_norm.weight",
+          "model.llm.layers.7.attn_sconv.weight",
+          "model.llm.layers.7.mlp.gate.weight",
+          "model.llm.layers.7.mlp_norm.weight",
+          "model.llm.layers.7.mlp_sconv.weight",
+          "model.llm.layers.8.attn.attn_sink_proj.weight",
+          "model.llm.layers.8.attn.gate_attn.weight",
+          "model.llm.layers.8.attn.k_sconv.weight",
+          "model.llm.layers.8.attn.q_l2_clip.scale",
+          "model.llm.layers.8.attn.rel_l2_clip.scale",
+          "model.llm.layers.8.attn.rel_logits_proj.proj",
+          "model.llm.layers.8.attn.v_sconv.weight",
+          "model.llm.layers.8.attn.wk_dv.weight",
+          "model.llm.layers.8.attn.wo_ud.weight",
+          "model.llm.layers.8.attn.wq_du.weight",
+          "model.llm.layers.8.attn.wr_du.weight",
+          "model.llm.layers.8.attn.wv_dv.weight",
+          "model.llm.layers.8.attn_norm.weight",
+          "model.llm.layers.8.attn_sconv.weight",
+          "model.llm.layers.8.mlp.gate.weight",
+          "model.llm.layers.8.mlp_norm.weight",
+          "model.llm.layers.8.mlp_sconv.weight",
+          "model.llm.layers.9.attn.attn_sink_proj.weight",
+          "model.llm.layers.9.attn.gate_attn.weight",
+          "model.llm.layers.9.attn.k_sconv.weight",
+          "model.llm.layers.9.attn.q_l2_clip.scale",
+          "model.llm.layers.9.attn.rel_l2_clip.scale",
+          "model.llm.layers.9.attn.rel_logits_proj.proj",
+          "model.llm.layers.9.attn.v_sconv.weight",
+          "model.llm.layers.9.attn.wk_dv.weight",
+          "model.llm.layers.9.attn.wo_ud.weight",
+          "model.llm.layers.9.attn.wq_du.weight",
+          "model.llm.layers.9.attn.wr_du.weight",
+          "model.llm.layers.9.attn.wv_dv.weight",
+          "model.llm.layers.9.attn_norm.weight",
+          "model.llm.layers.9.attn_sconv.weight",
+          "model.llm.layers.9.mlp.gate.weight",
+          "model.llm.layers.9.mlp_norm.weight",
+          "model.llm.layers.9.mlp_sconv.weight",
+          "model.llm.norm.weight",
+          "model.visual.backbone.embeddings.class_embedding",
+          "model.visual.backbone.embeddings.cls_position_embedding",
+          "model.visual.backbone.embeddings.patch_embedding.weight",
+          "model.visual.backbone.embeddings.position_embedding",
+          "model.visual.backbone.layers.0.ln_1.bias",
+          "model.visual.backbone.layers.0.ln_1.weight",
+          "model.visual.backbone.layers.0.ln_2.bias",
+          "model.visual.backbone.layers.0.ln_2.weight",
+          "model.visual.backbone.layers.0.ls_1.gamma",
+          "model.visual.backbone.layers.0.ls_2.gamma",
+          "model.visual.backbone.layers.0.mlp.c_fc.bias",
+          "model.visual.backbone.layers.0.mlp.c_fc.weight",
+          "model.visual.backbone.layers.0.mlp.c_proj.bias",
+          "model.visual.backbone.layers.0.mlp.c_proj.weight",
+          "model.visual.backbone.layers.0.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.0.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.0.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.0.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.0.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.0.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.0.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.0.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.1.ln_1.bias",
+          "model.visual.backbone.layers.1.ln_1.weight",
+          "model.visual.backbone.layers.1.ln_2.bias",
+          "model.visual.backbone.layers.1.ln_2.weight",
+          "model.visual.backbone.layers.1.ls_1.gamma",
+          "model.visual.backbone.layers.1.ls_2.gamma",
+          "model.visual.backbone.layers.1.mlp.c_fc.bias",
+          "model.visual.backbone.layers.1.mlp.c_fc.weight",
+          "model.visual.backbone.layers.1.mlp.c_proj.bias",
+          "model.visual.backbone.layers.1.mlp.c_proj.weight",
+          "model.visual.backbone.layers.1.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.1.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.1.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.1.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.1.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.1.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.1.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.1.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.10.ln_1.bias",
+          "model.visual.backbone.layers.10.ln_1.weight",
+          "model.visual.backbone.layers.10.ln_2.bias",
+          "model.visual.backbone.layers.10.ln_2.weight",
+          "model.visual.backbone.layers.10.ls_1.gamma",
+          "model.visual.backbone.layers.10.ls_2.gamma",
+          "model.visual.backbone.layers.10.mlp.c_fc.bias",
+          "model.visual.backbone.layers.10.mlp.c_fc.weight",
+          "model.visual.backbone.layers.10.mlp.c_proj.bias",
+          "model.visual.backbone.layers.10.mlp.c_proj.weight",
+          "model.visual.backbone.layers.10.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.10.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.10.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.10.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.10.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.10.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.10.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.10.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.11.ln_1.bias",
+          "model.visual.backbone.layers.11.ln_1.weight",
+          "model.visual.backbone.layers.11.ln_2.bias",
+          "model.visual.backbone.layers.11.ln_2.weight",
+          "model.visual.backbone.layers.11.ls_1.gamma",
+          "model.visual.backbone.layers.11.ls_2.gamma",
+          "model.visual.backbone.layers.11.mlp.c_fc.bias",
+          "model.visual.backbone.layers.11.mlp.c_fc.weight",
+          "model.visual.backbone.layers.11.mlp.c_proj.bias",
+          "model.visual.backbone.layers.11.mlp.c_proj.weight",
+          "model.visual.backbone.layers.11.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.11.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.11.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.11.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.11.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.11.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.11.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.11.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.12.ln_1.bias",
+          "model.visual.backbone.layers.12.ln_1.weight",
+          "model.visual.backbone.layers.12.ln_2.bias",
+          "model.visual.backbone.layers.12.ln_2.weight",
+          "model.visual.backbone.layers.12.ls_1.gamma",
+          "model.visual.backbone.layers.12.ls_2.gamma",
+          "model.visual.backbone.layers.12.mlp.c_fc.bias",
+          "model.visual.backbone.layers.12.mlp.c_fc.weight",
+          "model.visual.backbone.layers.12.mlp.c_proj.bias",
+          "model.visual.backbone.layers.12.mlp.c_proj.weight",
+          "model.visual.backbone.layers.12.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.12.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.12.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.12.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.12.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.12.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.12.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.12.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.13.ln_1.bias",
+          "model.visual.backbone.layers.13.ln_1.weight",
+          "model.visual.backbone.layers.13.ln_2.bias",
+          "model.visual.backbone.layers.13.ln_2.weight",
+          "model.visual.backbone.layers.13.ls_1.gamma",
+          "model.visual.backbone.layers.13.ls_2.gamma",
+          "model.visual.backbone.layers.13.mlp.c_fc.bias",
+          "model.visual.backbone.layers.13.mlp.c_fc.weight",
+          "model.visual.backbone.layers.13.mlp.c_proj.bias",
+          "model.visual.backbone.layers.13.mlp.c_proj.weight",
+          "model.visual.backbone.layers.13.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.13.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.13.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.13.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.13.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.13.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.13.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.13.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.14.ln_1.bias",
+          "model.visual.backbone.layers.14.ln_1.weight",
+          "model.visual.backbone.layers.14.ln_2.bias",
+          "model.visual.backbone.layers.14.ln_2.weight",
+          "model.visual.backbone.layers.14.ls_1.gamma",
+          "model.visual.backbone.layers.14.ls_2.gamma",
+          "model.visual.backbone.layers.14.mlp.c_fc.bias",
+          "model.visual.backbone.layers.14.mlp.c_fc.weight",
+          "model.visual.backbone.layers.14.mlp.c_proj.bias",
+          "model.visual.backbone.layers.14.mlp.c_proj.weight",
+          "model.visual.backbone.layers.14.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.14.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.14.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.14.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.14.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.14.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.14.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.14.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.15.ln_1.bias",
+          "model.visual.backbone.layers.15.ln_1.weight",
+          "model.visual.backbone.layers.15.ln_2.bias",
+          "model.visual.backbone.layers.15.ln_2.weight",
+          "model.visual.backbone.layers.15.ls_1.gamma",
+          "model.visual.backbone.layers.15.ls_2.gamma",
+          "model.visual.backbone.layers.15.mlp.c_fc.bias",
+          "model.visual.backbone.layers.15.mlp.c_fc.weight",
+          "model.visual.backbone.layers.15.mlp.c_proj.bias",
+          "model.visual.backbone.layers.15.mlp.c_proj.weight",
+          "model.visual.backbone.layers.15.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.15.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.15.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.15.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.15.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.15.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.15.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.15.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.16.ln_1.bias",
+          "model.visual.backbone.layers.16.ln_1.weight",
+          "model.visual.backbone.layers.16.ln_2.bias",
+          "model.visual.backbone.layers.16.ln_2.weight",
+          "model.visual.backbone.layers.16.ls_1.gamma",
+          "model.visual.backbone.layers.16.ls_2.gamma",
+          "model.visual.backbone.layers.16.mlp.c_fc.bias",
+          "model.visual.backbone.layers.16.mlp.c_fc.weight",
+          "model.visual.backbone.layers.16.mlp.c_proj.bias",
+          "model.visual.backbone.layers.16.mlp.c_proj.weight",
+          "model.visual.backbone.layers.16.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.16.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.16.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.16.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.16.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.16.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.16.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.16.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.17.ln_1.bias",
+          "model.visual.backbone.layers.17.ln_1.weight",
+          "model.visual.backbone.layers.17.ln_2.bias",
+          "model.visual.backbone.layers.17.ln_2.weight",
+          "model.visual.backbone.layers.17.ls_1.gamma",
+          "model.visual.backbone.layers.17.ls_2.gamma",
+          "model.visual.backbone.layers.17.mlp.c_fc.bias",
+          "model.visual.backbone.layers.17.mlp.c_fc.weight",
+          "model.visual.backbone.layers.17.mlp.c_proj.bias",
+          "model.visual.backbone.layers.17.mlp.c_proj.weight",
+          "model.visual.backbone.layers.17.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.17.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.17.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.17.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.17.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.17.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.17.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.17.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.18.ln_1.bias",
+          "model.visual.backbone.layers.18.ln_1.weight",
+          "model.visual.backbone.layers.18.ln_2.bias",
+          "model.visual.backbone.layers.18.ln_2.weight",
+          "model.visual.backbone.layers.18.ls_1.gamma",
+          "model.visual.backbone.layers.18.ls_2.gamma",
+          "model.visual.backbone.layers.18.mlp.c_fc.bias",
+          "model.visual.backbone.layers.18.mlp.c_fc.weight",
+          "model.visual.backbone.layers.18.mlp.c_proj.bias",
+          "model.visual.backbone.layers.18.mlp.c_proj.weight",
+          "model.visual.backbone.layers.18.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.18.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.18.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.18.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.18.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.18.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.18.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.18.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.19.ln_1.bias",
+          "model.visual.backbone.layers.19.ln_1.weight",
+          "model.visual.backbone.layers.19.ln_2.bias",
+          "model.visual.backbone.layers.19.ln_2.weight",
+          "model.visual.backbone.layers.19.ls_1.gamma",
+          "model.visual.backbone.layers.19.ls_2.gamma",
+          "model.visual.backbone.layers.19.mlp.c_fc.bias",
+          "model.visual.backbone.layers.19.mlp.c_fc.weight",
+          "model.visual.backbone.layers.19.mlp.c_proj.bias",
+          "model.visual.backbone.layers.19.mlp.c_proj.weight",
+          "model.visual.backbone.layers.19.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.19.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.19.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.19.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.19.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.19.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.19.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.19.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.2.ln_1.bias",
+          "model.visual.backbone.layers.2.ln_1.weight",
+          "model.visual.backbone.layers.2.ln_2.bias",
+          "model.visual.backbone.layers.2.ln_2.weight",
+          "model.visual.backbone.layers.2.ls_1.gamma",
+          "model.visual.backbone.layers.2.ls_2.gamma",
+          "model.visual.backbone.layers.2.mlp.c_fc.bias",
+          "model.visual.backbone.layers.2.mlp.c_fc.weight",
+          "model.visual.backbone.layers.2.mlp.c_proj.bias",
+          "model.visual.backbone.layers.2.mlp.c_proj.weight",
+          "model.visual.backbone.layers.2.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.2.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.2.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.2.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.2.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.2.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.2.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.2.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.20.ln_1.bias",
+          "model.visual.backbone.layers.20.ln_1.weight",
+          "model.visual.backbone.layers.20.ln_2.bias",
+          "model.visual.backbone.layers.20.ln_2.weight",
+          "model.visual.backbone.layers.20.ls_1.gamma",
+          "model.visual.backbone.layers.20.ls_2.gamma",
+          "model.visual.backbone.layers.20.mlp.c_fc.bias",
+          "model.visual.backbone.layers.20.mlp.c_fc.weight",
+          "model.visual.backbone.layers.20.mlp.c_proj.bias",
+          "model.visual.backbone.layers.20.mlp.c_proj.weight",
+          "model.visual.backbone.layers.20.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.20.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.20.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.20.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.20.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.20.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.20.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.20.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.21.ln_1.bias",
+          "model.visual.backbone.layers.21.ln_1.weight",
+          "model.visual.backbone.layers.21.ln_2.bias",
+          "model.visual.backbone.layers.21.ln_2.weight",
+          "model.visual.backbone.layers.21.ls_1.gamma",
+          "model.visual.backbone.layers.21.ls_2.gamma",
+          "model.visual.backbone.layers.21.mlp.c_fc.bias",
+          "model.visual.backbone.layers.21.mlp.c_fc.weight",
+          "model.visual.backbone.layers.21.mlp.c_proj.bias",
+          "model.visual.backbone.layers.21.mlp.c_proj.weight",
+          "model.visual.backbone.layers.21.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.21.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.21.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.21.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.21.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.21.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.21.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.21.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.22.ln_1.bias",
+          "model.visual.backbone.layers.22.ln_1.weight",
+          "model.visual.backbone.layers.22.ln_2.bias",
+          "model.visual.backbone.layers.22.ln_2.weight",
+          "model.visual.backbone.layers.22.ls_1.gamma",
+          "model.visual.backbone.layers.22.ls_2.gamma",
+          "model.visual.backbone.layers.22.mlp.c_fc.bias",
+          "model.visual.backbone.layers.22.mlp.c_fc.weight",
+          "model.visual.backbone.layers.22.mlp.c_proj.bias",
+          "model.visual.backbone.layers.22.mlp.c_proj.weight",
+          "model.visual.backbone.layers.22.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.22.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.22.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.22.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.22.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.22.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.22.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.22.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.3.ln_1.bias",
+          "model.visual.backbone.layers.3.ln_1.weight",
+          "model.visual.backbone.layers.3.ln_2.bias",
+          "model.visual.backbone.layers.3.ln_2.weight",
+          "model.visual.backbone.layers.3.ls_1.gamma",
+          "model.visual.backbone.layers.3.ls_2.gamma",
+          "model.visual.backbone.layers.3.mlp.c_fc.bias",
+          "model.visual.backbone.layers.3.mlp.c_fc.weight",
+          "model.visual.backbone.layers.3.mlp.c_proj.bias",
+          "model.visual.backbone.layers.3.mlp.c_proj.weight",
+          "model.visual.backbone.layers.3.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.3.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.3.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.3.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.3.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.3.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.3.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.3.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.4.ln_1.bias",
+          "model.visual.backbone.layers.4.ln_1.weight",
+          "model.visual.backbone.layers.4.ln_2.bias",
+          "model.visual.backbone.layers.4.ln_2.weight",
+          "model.visual.backbone.layers.4.ls_1.gamma",
+          "model.visual.backbone.layers.4.ls_2.gamma",
+          "model.visual.backbone.layers.4.mlp.c_fc.bias",
+          "model.visual.backbone.layers.4.mlp.c_fc.weight",
+          "model.visual.backbone.layers.4.mlp.c_proj.bias",
+          "model.visual.backbone.layers.4.mlp.c_proj.weight",
+          "model.visual.backbone.layers.4.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.4.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.4.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.4.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.4.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.4.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.4.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.4.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.5.ln_1.bias",
+          "model.visual.backbone.layers.5.ln_1.weight",
+          "model.visual.backbone.layers.5.ln_2.bias",
+          "model.visual.backbone.layers.5.ln_2.weight",
+          "model.visual.backbone.layers.5.ls_1.gamma",
+          "model.visual.backbone.layers.5.ls_2.gamma",
+          "model.visual.backbone.layers.5.mlp.c_fc.bias",
+          "model.visual.backbone.layers.5.mlp.c_fc.weight",
+          "model.visual.backbone.layers.5.mlp.c_proj.bias",
+          "model.visual.backbone.layers.5.mlp.c_proj.weight",
+          "model.visual.backbone.layers.5.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.5.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.5.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.5.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.5.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.5.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.5.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.5.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.6.ln_1.bias",
+          "model.visual.backbone.layers.6.ln_1.weight",
+          "model.visual.backbone.layers.6.ln_2.bias",
+          "model.visual.backbone.layers.6.ln_2.weight",
+          "model.visual.backbone.layers.6.ls_1.gamma",
+          "model.visual.backbone.layers.6.ls_2.gamma",
+          "model.visual.backbone.layers.6.mlp.c_fc.bias",
+          "model.visual.backbone.layers.6.mlp.c_fc.weight",
+          "model.visual.backbone.layers.6.mlp.c_proj.bias",
+          "model.visual.backbone.layers.6.mlp.c_proj.weight",
+          "model.visual.backbone.layers.6.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.6.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.6.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.6.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.6.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.6.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.6.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.6.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.7.ln_1.bias",
+          "model.visual.backbone.layers.7.ln_1.weight",
+          "model.visual.backbone.layers.7.ln_2.bias",
+          "model.visual.backbone.layers.7.ln_2.weight",
+          "model.visual.backbone.layers.7.ls_1.gamma",
+          "model.visual.backbone.layers.7.ls_2.gamma",
+          "model.visual.backbone.layers.7.mlp.c_fc.bias",
+          "model.visual.backbone.layers.7.mlp.c_fc.weight",
+          "model.visual.backbone.layers.7.mlp.c_proj.bias",
+          "model.visual.backbone.layers.7.mlp.c_proj.weight",
+          "model.visual.backbone.layers.7.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.7.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.7.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.7.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.7.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.7.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.7.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.7.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.8.ln_1.bias",
+          "model.visual.backbone.layers.8.ln_1.weight",
+          "model.visual.backbone.layers.8.ln_2.bias",
+          "model.visual.backbone.layers.8.ln_2.weight",
+          "model.visual.backbone.layers.8.ls_1.gamma",
+          "model.visual.backbone.layers.8.ls_2.gamma",
+          "model.visual.backbone.layers.8.mlp.c_fc.bias",
+          "model.visual.backbone.layers.8.mlp.c_fc.weight",
+          "model.visual.backbone.layers.8.mlp.c_proj.bias",
+          "model.visual.backbone.layers.8.mlp.c_proj.weight",
+          "model.visual.backbone.layers.8.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.8.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.8.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.8.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.8.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.8.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.8.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.8.self_attn.v_proj.weight",
+          "model.visual.backbone.layers.9.ln_1.bias",
+          "model.visual.backbone.layers.9.ln_1.weight",
+          "model.visual.backbone.layers.9.ln_2.bias",
+          "model.visual.backbone.layers.9.ln_2.weight",
+          "model.visual.backbone.layers.9.ls_1.gamma",
+          "model.visual.backbone.layers.9.ls_2.gamma",
+          "model.visual.backbone.layers.9.mlp.c_fc.bias",
+          "model.visual.backbone.layers.9.mlp.c_fc.weight",
+          "model.visual.backbone.layers.9.mlp.c_proj.bias",
+          "model.visual.backbone.layers.9.mlp.c_proj.weight",
+          "model.visual.backbone.layers.9.self_attn.k_proj.bias",
+          "model.visual.backbone.layers.9.self_attn.k_proj.weight",
+          "model.visual.backbone.layers.9.self_attn.out_proj.bias",
+          "model.visual.backbone.layers.9.self_attn.out_proj.weight",
+          "model.visual.backbone.layers.9.self_attn.q_proj.bias",
+          "model.visual.backbone.layers.9.self_attn.q_proj.weight",
+          "model.visual.backbone.layers.9.self_attn.v_proj.bias",
+          "model.visual.backbone.layers.9.self_attn.v_proj.weight",
+          "model.visual.backbone.ln_pre.bias",
+          "model.visual.backbone.ln_pre.weight",
+          "model.visual.proj_in.weight",
+          "model.visual.proj_out.weight"
+        ],
+        "weight_block_shape": [],
+        "weight_transform": "cast"
+      },
+      {
+        "compute_dtype": "fp8",
+        "output_dtype": "float8_e4m3fnuz",
+        "required_activation_quantization": {
+          "format": "float8_e4m3fnuz",
+          "granularity": "per_token",
+          "scale_dtype": "float32",
+          "rounding": "nearest"
+        },
+        "tensors": [
+          "model.llm.layers.0.mlp.experts.w13_weight",
+          "model.llm.layers.0.mlp.experts.w2_weight",
+          "model.llm.layers.1.mlp.experts.w13_weight",
+          "model.llm.layers.1.mlp.experts.w2_weight",
+          "model.llm.layers.10.mlp.experts.w13_weight",
+          "model.llm.layers.10.mlp.experts.w2_weight",
+          "model.llm.layers.11.mlp.experts.w13_weight",
+          "model.llm.layers.11.mlp.experts.w2_weight",
+          "model.llm.layers.12.mlp.experts.w13_weight",
+          "model.llm.layers.12.mlp.experts.w2_weight",
+          "model.llm.layers.13.mlp.experts.w13_weight",
+          "model.llm.layers.13.mlp.experts.w2_weight",
+          "model.llm.layers.14.mlp.experts.w13_weight",
+          "model.llm.layers.14.mlp.experts.w2_weight",
+          "model.llm.layers.15.mlp.experts.w13_weight",
+          "model.llm.layers.15.mlp.experts.w2_weight",
+          "model.llm.layers.16.mlp.experts.w13_weight",
+          "model.llm.layers.16.mlp.experts.w2_weight",
+          "model.llm.layers.17.mlp.experts.w13_weight",
+          "model.llm.layers.17.mlp.experts.w2_weight",
+          "model.llm.layers.18.mlp.experts.w13_weight",
+          "model.llm.layers.18.mlp.experts.w2_weight",
+          "model.llm.layers.19.mlp.experts.w13_weight",
+          "model.llm.layers.19.mlp.experts.w2_weight",
+          "model.llm.layers.2.mlp.experts.w13_weight",
+          "model.llm.layers.2.mlp.experts.w2_weight",
+          "model.llm.layers.20.mlp.experts.w13_weight",
+          "model.llm.layers.20.mlp.experts.w2_weight",
+          "model.llm.layers.21.mlp.experts.w13_weight",
+          "model.llm.layers.21.mlp.experts.w2_weight",
+          "model.llm.layers.22.mlp.experts.w13_weight",
+          "model.llm.layers.22.mlp.experts.w2_weight",
+          "model.llm.layers.23.mlp.experts.w13_weight",
+          "model.llm.layers.23.mlp.experts.w2_weight",
+          "model.llm.layers.24.mlp.experts.w13_weight",
+          "model.llm.layers.24.mlp.experts.w2_weight",
+          "model.llm.layers.25.mlp.experts.w13_weight",
+          "model.llm.layers.25.mlp.experts.w2_weight",
+          "model.llm.layers.26.mlp.experts.w13_weight",
+          "model.llm.layers.26.mlp.experts.w2_weight",
+          "model.llm.layers.27.mlp.experts.w13_weight",
+          "model.llm.layers.27.mlp.experts.w2_weight",
+          "model.llm.layers.28.mlp.experts.w13_weight",
+          "model.llm.layers.28.mlp.experts.w2_weight",
+          "model.llm.layers.29.mlp.experts.w13_weight",
+          "model.llm.layers.29.mlp.experts.w2_weight",
+          "model.llm.layers.3.mlp.experts.w13_weight",
+          "model.llm.layers.3.mlp.experts.w2_weight",
+          "model.llm.layers.30.mlp.experts.w13_weight",
+          "model.llm.layers.30.mlp.experts.w2_weight",
+          "model.llm.layers.31.mlp.experts.w13_weight",
+          "model.llm.layers.31.mlp.experts.w2_weight",
+          "model.llm.layers.32.mlp.experts.w13_weight",
+          "model.llm.layers.32.mlp.experts.w2_weight",
+          "model.llm.layers.33.mlp.experts.w13_weight",
+          "model.llm.layers.33.mlp.experts.w2_weight",
+          "model.llm.layers.34.mlp.experts.w13_weight",
+          "model.llm.layers.34.mlp.experts.w2_weight",
+          "model.llm.layers.35.mlp.experts.w13_weight",
+          "model.llm.layers.35.mlp.experts.w2_weight",
+          "model.llm.layers.36.mlp.experts.w13_weight",
+          "model.llm.layers.36.mlp.experts.w2_weight",
+          "model.llm.layers.37.mlp.experts.w13_weight",
+          "model.llm.layers.37.mlp.experts.w2_weight",
+          "model.llm.layers.38.mlp.experts.w13_weight",
+          "model.llm.layers.38.mlp.experts.w2_weight",
+          "model.llm.layers.39.mlp.experts.w13_weight",
+          "model.llm.layers.39.mlp.experts.w2_weight",
+          "model.llm.layers.4.mlp.experts.w13_weight",
+          "model.llm.layers.4.mlp.experts.w2_weight",
+          "model.llm.layers.40.mlp.experts.w13_weight",
+          "model.llm.layers.40.mlp.experts.w2_weight",
+          "model.llm.layers.41.mlp.experts.w13_weight",
+          "model.llm.layers.41.mlp.experts.w2_weight",
+          "model.llm.layers.42.mlp.experts.w13_weight",
+          "model.llm.layers.42.mlp.experts.w2_weight",
+          "model.llm.layers.43.mlp.experts.w13_weight",
+          "model.llm.layers.43.mlp.experts.w2_weight",
+          "model.llm.layers.44.mlp.experts.w13_weight",
+          "model.llm.layers.44.mlp.experts.w2_weight",
+          "model.llm.layers.45.mlp.experts.w13_weight",
+          "model.llm.layers.45.mlp.experts.w2_weight",
+          "model.llm.layers.46.mlp.experts.w13_weight",
+          "model.llm.layers.46.mlp.experts.w2_weight",
+          "model.llm.layers.47.mlp.experts.w13_weight",
+          "model.llm.layers.47.mlp.experts.w2_weight",
+          "model.llm.layers.5.mlp.experts.w13_weight",
+          "model.llm.layers.5.mlp.experts.w2_weight",
+          "model.llm.layers.6.mlp.experts.w13_weight",
+          "model.llm.layers.6.mlp.experts.w2_weight",
+          "model.llm.layers.7.mlp.experts.w13_weight",
+          "model.llm.layers.7.mlp.experts.w2_weight",
+          "model.llm.layers.8.mlp.experts.w13_weight",
+          "model.llm.layers.8.mlp.experts.w2_weight",
+          "model.llm.layers.9.mlp.experts.w13_weight",
+          "model.llm.layers.9.mlp.experts.w2_weight"
+        ],
+        "weight_block_shape": [],
+        "weight_transform": "row_fp8_e4m3fnuz_max_240"
+      },
+      {
+        "compute_dtype": "bfloat16",
+        "fwd_output_dtype": "float32",
+        "output_dtype": "bfloat16",
+        "required_activation_quantization": null,
+        "tensors": [
+          "model.llm.unembed.weight"
+        ],
+        "weight_block_shape": [],
+        "weight_transform": "cast"
+      }
+    ]
+  },
+  "mtp": {
+    "parameter_groups": [
+      {
+        "compute_dtype": "bfloat16",
+        "output_dtype": "bfloat16",
+        "required_activation_quantization": null,
+        "tensors": [
+          "fuse_weight",
+          "layers.0.dflare_norm.weight",
+          "layers.0.input_layernorm.weight",
+          "layers.0.mlp.down_proj.weight",
+          "layers.0.mlp.gate_proj.weight",
+          "layers.0.mlp.up_proj.weight",
+          "layers.0.post_attention_layernorm.weight",
+          "layers.0.self_attn.k_norm.weight",
+          "layers.0.self_attn.k_norm_ctx.weight",
+          "layers.0.self_attn.k_proj.weight",
+          "layers.0.self_attn.k_proj_ctx.weight",
+          "layers.0.self_attn.o_proj.weight",
+          "layers.0.self_attn.q_norm.weight",
+          "layers.0.self_attn.q_proj.weight",
+          "layers.0.self_attn.v_proj.weight",
+          "layers.0.self_attn.v_proj_ctx.weight",
+          "layers.1.dflare_norm.weight",
+          "layers.1.input_layernorm.weight",
+          "layers.1.mlp.down_proj.weight",
+          "layers.1.mlp.gate_proj.weight",
+          "layers.1.mlp.up_proj.weight",
+          "layers.1.post_attention_layernorm.weight",
+          "layers.1.self_attn.k_norm.weight",
+          "layers.1.self_attn.k_norm_ctx.weight",
+          "layers.1.self_attn.k_proj.weight",
+          "layers.1.self_attn.k_proj_ctx.weight",
+          "layers.1.self_attn.o_proj.weight",
+          "layers.1.self_attn.q_norm.weight",
+          "layers.1.self_attn.q_proj.weight",
+          "layers.1.self_attn.v_proj.weight",
+          "layers.1.self_attn.v_proj_ctx.weight",
+          "layers.2.dflare_norm.weight",
+          "layers.2.input_layernorm.weight",
+          "layers.2.mlp.down_proj.weight",
+          "layers.2.mlp.gate_proj.weight",
+          "layers.2.mlp.up_proj.weight",
+          "layers.2.post_attention_layernorm.weight",
+          "layers.2.self_attn.k_norm.weight",
+          "layers.2.self_attn.k_norm_ctx.weight",
+          "layers.2.self_attn.k_proj.weight",
+          "layers.2.self_attn.k_proj_ctx.weight",
+          "layers.2.self_attn.o_proj.weight",
+          "layers.2.self_attn.q_norm.weight",
+          "layers.2.self_attn.q_proj.weight",
+          "layers.2.self_attn.v_proj.weight",
+          "layers.2.self_attn.v_proj_ctx.weight",
+          "layers.3.dflare_norm.weight",
+          "layers.3.input_layernorm.weight",
+          "layers.3.mlp.down_proj.weight",
+          "layers.3.mlp.gate_proj.weight",
+          "layers.3.mlp.up_proj.weight",
+          "layers.3.post_attention_layernorm.weight",
+          "layers.3.self_attn.k_norm.weight",
+          "layers.3.self_attn.k_norm_ctx.weight",
+          "layers.3.self_attn.k_proj.weight",
+          "layers.3.self_attn.k_proj_ctx.weight",
+          "layers.3.self_attn.o_proj.weight",
+          "layers.3.self_attn.q_norm.weight",
+          "layers.3.self_attn.q_proj.weight",
+          "layers.3.self_attn.v_proj.weight",
+          "layers.3.self_attn.v_proj_ctx.weight",
+          "layers.4.dflare_norm.weight",
+          "layers.4.input_layernorm.weight",
+          "layers.4.mlp.down_proj.weight",
+          "layers.4.mlp.gate_proj.weight",
+          "layers.4.mlp.up_proj.weight",
+          "layers.4.post_attention_layernorm.weight",
+          "layers.4.self_attn.k_norm.weight",
+          "layers.4.self_attn.k_norm_ctx.weight",
+          "layers.4.self_attn.k_proj.weight",
+          "layers.4.self_attn.k_proj_ctx.weight",
+          "layers.4.self_attn.o_proj.weight",
+          "layers.4.self_attn.q_norm.weight",
+          "layers.4.self_attn.q_proj.weight",
+          "layers.4.self_attn.v_proj.weight",
+          "layers.4.self_attn.v_proj_ctx.weight",
+          "markov_head.dflare_norm.weight",
+          "markov_head.hidden_out.bias",
+          "markov_head.hidden_out.weight",
+          "markov_head.markov_w1.weight",
+          "markov_head.rnn_proj.bias",
+          "markov_head.rnn_proj.weight",
+          "mask_embed",
+          "norm.weight"
+        ],
+        "weight_block_shape": [],
+        "weight_transform": "cast"
+      }
+    ]
+  },
+  "quant_method": "inkling2",
+  "expert_weight_quantization": {
+    "format": "float8_e4m3fnuz",
+    "granularity": "per_output_channel",
+    "scale_dtype": "float32",
+    "scale_tensor_suffix": "_scale"
+  }
+}

model-00001-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7fad470eaec8bad8f1dc777d99568acb3d1ed565955220983defc4dc3dacdd4e
+size 14495803536

model-00002-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a5d1e2f0446f6862c67a4d0745e9c34f8e077836fb2d6f47c47cb39ccc9863c
+size 12444708336

model-00003-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d75382147057d2b48ab9a3cc526b3042cf2957056c378987fc86db432ca6ca61
+size 12445298408

model-00004-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b12cdfb7fb1e9690b594403b6a7b7e8f701e842a1f4c076a9854bc5cadca0e83
+size 12444727000

model-00005-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ae0308a826ab439da3acf2fd6318ccc3888eeb11b6446e89fecebd26be40985
+size 12444708344

model-00006-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0e016445e5f2a1ef0308ee0fa8855ca1938d79a89a653e69893bb078ca77023
+size 12444708344

model-00007-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43a0fbe2b0cec6eb13863636ecee97c11eb71648747a98ce2fd1a991cc4e7e0d
+size 12444708344

model-00008-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11c6daa9875487c02fd4245246a368a7ec35c2f26542b4666ff617d446c4c5e1
+size 12444708344

model-00009-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b75a5ae0d5c9ea81dc72c6c6df74b0c343e4c90d6a6dcf3697a6e433cfefc831
+size 12445298408

model-00010-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bd169e1a157d2cbd780b62933eef58c454735811653a11b1236e66529889cc7
+size 12444727000

model-00011-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98b2bd93ffd7f3d81771059d15032ce442b6c700fb73322fa4a3be87d67d4986
+size 12444708344

model-00012-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f4964173c9ff4e1860867f88a098bc775447f2866da0bd3b4fbaa6c74b0e772
+size 12444708336

model-00013-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e1d8647d5c88a35caa470b84d63762ec1318fed6a29f13e435b2997e61326fc
+size 12444708336

model-00014-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9484aa75dc39578d66f9a6f6cb42f190877b6f6a8ba9902f2ac76e890c01a7b0
+size 12444708344

model-00015-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1298864e8bd1e0ebaa7d0371df01042efb54e4a46b50d81b1c94e2d0823e58f9
+size 12444708344

model-00016-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:746a09951ce0083ca26747915125318e9bdcd27454544a46aeb29278f5f7ef17
+size 12445298408

model-00017-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35afebd2497665036f6720a044ba97ac63788f6a38a50db612e4262d3df873fc
+size 12444727000

model-00018-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbba2995112f808268e2026b3516d7af710dbce59f7a1ce4e2f71babac5158af
+size 12444708344

model-00019-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69c113a58920c9ed629811be3346e5bd748b471e0abb39bd39f4e1085658d60b
+size 12444708344

model-00020-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fd40234ebafcfbb6f79032f3cc75e01b22a6024890f39196c233af96d1329a4
+size 12444708344

model-00021-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ade09a0fb2bc900fedccd08b22b64c97f8887155b96c8029e49930f3bd325fe4
+size 12444708344

model-00022-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f0b18702437737f03f72f1f47ac23e409c18c8eea44f8cd9e439bd914d4c4aa
+size 12445298408

model-00023-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4997343c248395463e6818f247bedbe5a9569f055b233646d16a8755a702978b
+size 12444726984

model-00024-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e4757b0b9fffbb32cd6b89910db636462e52f3229131073539d7293471b09ca
+size 12444708336

model-00025-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73fffa315775b2bea3adb506655540274aed45b64bf2390dca1cd1333f1a839c
+size 12444708344

model-00026-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fb4572a2ef1f76188d29bbe73dda86e6bf099b0a24b6a77a418b7dc944cedff
+size 12444708344

model-00027-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32b72d43d105895068ac9b50428f9e942067d52b23bef7aade025733ad3701ea
+size 12444708344

model-00028-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84e235753a6356a1047d8e7aea875d8da071bd995fb82ee844c276b8015d2d46
+size 12444708344

model-00029-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:960cae600512dcf5766d7ff55bbed36365ef89c167e7a4871ab17138dd1883fc
+size 12445298408

model-00030-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a2a0338f4b22ed77dea801159a45f859c7faa5ebea0d53085aa159aa40b1252
+size 12444727000

model-00031-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecdce253aca477d9cce056495d6545f479befda55b352bff75cee3aa75c7c3a6
+size 12444708344

model-00032-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b92c8cda5b4087c5595a32cc2351b656a59394a874f31e36de467ff8e370a32
+size 12444708344

model-00033-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4b9f6d5d15e943ef3e3bbc5502d5f921f69bb5d74aeb7e7b0f678b455a2ac9a
+size 12444708344

model-00034-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bb41c901da35268defbc985c0161847faeff4aebadeb08fc5294c5cb4fb7a10
+size 12444708336

model-00035-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64fee61ca6cd479e79ee4a30e460606edf4ac364bddfba90cc25ee8ed62f3e5c
+size 12444708336

model-00036-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0cc7aaf0c72a31d321076526ff28fa67c2343511dcfe8245e33804f819a4315
+size 12445298408

model-00037-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96c6362e84341cfac582dbae843d70f252ae0ce1fb88bd457b4c02c74ae1e5a2
+size 12444727000

model-00038-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31ceccfecc80ad5c3bebf9b984863e9f982b2510b947997dcb842840103fb591
+size 12444708344

model-00039-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7ffe1096223475b6d699bdac0cf8188120124c2be6ed55ff7532832b5afe948
+size 12444708344

model-00040-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9bbde667c713237de6f545e04717c0190902355f172762df0113e5eb512140f
+size 12444708344

model-00041-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4226864e33dfdbf378440fec0fc41b662f8ebfa0c2ab38c462f82cf3d6b8b95b
+size 12444708344

model-00042-of-00048.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0eec71b24560650ef0faf4a0707ef0e47d001b09cc2c84f93749f845e3990b0
+size 12445298408