Upload folder using huggingface_hub

pass3-hope/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Z.AI Co., Ltd
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pass3-hope/README.md

@@ -0,0 +1,136 @@
+---
+license: mit
+base_model: zai-org/GLM-5.3-Flash
+tags: [reap, moe, pruning, glm5_next, jetson, thor]
+---
+
+# GLM-5.3-Flash REAP-50 (FP8)
+
+50% of routed experts removed with **REAP**
+(Router-weighted Expert Activation Pruning, arXiv:2510.13999), calibrated on a
+permissively-licensed multi-domain corpus that includes real image-text pairs.
+
+| | |
+|---|---|
+| Base | `zai-org/GLM-5.3-Flash` (MIT, FP8 E4M3, 128x128 block scales) |
+| Experts | 288 -> 144 per layer, top-8 routing unchanged |
+| Size | 160.6 GiB (FP8) |
+| Healed | yes |
+| MTP block | excluded (see below) |
+
+## Why FP8 and not BF16
+
+The upstream release is **FP8**, not BF16. Routed experts are stored per-expert with their own
+`weight_scale_inv` block scales, so pruning is deleting whole tensors - **lossless on every
+retained weight**. The 642 GB BF16 repo elsewhere on the Hub is a dequantised upcast carrying
+no additional information.
+
+## Calibration
+
+Mixture weighted for a coding/agentic model that stays empirically grounded: agentic 24%,
+code 21%, math 15%, multimodal 15%, science+bio 10%, finance 8%, ballast 7%.
+Permissive licences only, so this checkpoint keeps the base model's MIT lineage.
+
+Vision is first-class: the vision tower contains no MoE and is untouched, but image tokens
+route through the same expert pool as text, so text-only calibration would have deleted
+vision-serving experts with certainty. Real image-text pairs were asserted present.
+
+## Evaluation status: NONE
+
+**This checkpoint has not been evaluated.** No benchmark has been run against it - not coding,
+not agentic, not vision, not knowledge. What has been verified is *structural*: expert counts
+match the config, routers are sliced to the retained set, every tensor loads, the vision tower
+is untouched. The MTP block at layer 45 is cleanly absent.
+
+The pruning itself measured **1.34x better than random** at retaining expert output
+contribution (saliency mass 0.669 against 0.50 for random pruning at the same ratio). That says
+the criterion selected well. It does **not** say the model is good.
+
+Treat this as a research artifact pending evaluation, not a drop-in replacement.
+
+## What this prune costs, and what it does not
+
+REAP ranks experts on saliency pooled across the calibration mixture, so the cost is **not**
+spread evenly across capabilities. Measured on this checkpoint - the fraction of the expert
+output each domain actually relies on that survived the prune:
+
+| domain | retained |
+|---|---|
+| agentic / tool use | 0.747 |
+| code | 0.728 |
+| science | 0.720 |
+| maths | 0.713 |
+| vision (image-text) | 0.682 |
+| finance | 0.651 |
+| general / ballast | **0.487** |
+
+**The damage is concentrated where retrieval can repair it.** Generic factual ballast is the
+worst-retained bucket, and it is also the single most RAG-recoverable capability: a markdown
+corpus and a retriever substitute for memorised trivia almost perfectly. The best-retained
+buckets - agentic behaviour, code, science, maths - are the ones retrieval **cannot** restore,
+because you cannot retrieve your way to reasoning, tool use, or code synthesis.
+
+If you are deploying this, pair it with retrieval. That is not a workaround for a defect; it is
+the shape the trade was made in. A uniformly-quantised model of the same footprint spends its
+degradation budget evenly, including on the capabilities retrieval cannot give back.
+
+Two honest caveats. Low retention can also mean a domain uses experts *diffusely* rather than
+that its capability was removed - generic web text spreads across many experts while REAP keeps
+concentrated ones. And the per-domain spread was a **consequence** of the calibration mixture,
+not a targeted design: this is mixture-shaped preservation, not surgery.
+
+## Method: strengths and gaps
+
+**Where this is above common practice**
+
+- Calibration is 0.0M tokens across seven use-case-weighted domains, against the 128-512 generic
+  samples typical of pruning and quantisation work.
+- **Real image-text pairs** in calibration, not text descriptions of images. Measured effect:
+  vision retained 0.682, at the cross-domain mean. Text-only calibration deletes
+  vision-serving experts with near-certainty.
+- Saliency capture was **verified against the model source** - the router gate excludes
+  `e_score_correction_bias`, which is the detail most implementations get silently wrong.
+- Per-token router scores were cached, making post-prune routing exactly replayable offline.
+- The healing correction was **measured, not derived**. The standard first-moment estimator
+  over-corrects by ~30% on this architecture because it ignores `norm_topk_prob`
+  renormalisation; that error was found and fixed only because of the router cache.
+- The calibration budget was validated by a split-half stopping rule on retained saliency mass,
+  rather than assumed.
+
+**Where it falls short, stated plainly**
+
+- **Healing rescales experts; it does not retrain them.** The per-expert coefficients are the best
+  a *fixed rescaling* can do, and that ceiling is now measured rather than assumed: 96.6% of an
+  expert's output energy is token-dependent residual rather than its mean, so the ~0.27 residual
+  that remains is information deleted with the pruned experts, not error a better fit could
+  remove. Recovering it means layer-local distillation, which needs a backward pass that does not
+  fit the hardware this was built on.
+- **No expert merging** (REAM/EEP), rejected on cost.
+- **Evaluation is teacher-forced dNLL and top-1 agreement, not generative benchmarks.** That is
+  the largest gap: dNLL does not fully predict agentic or coding capability, which is what this
+  model is for. Generation from the unpruned teacher costs ~110 s/token on one device and was
+  out of reach.
+- **Long context is uncalibrated.** Calibration sequences were capped at 2048 tokens against a
+  1M-token context window.
+- The calibration mixture was specified in *samples* but acts in *tokens*; document lengths
+  differ by more than 10x, so realised domain weights differed substantially from intended.
+- A single prune ratio was materialised. No 40/50/60 ablation with evaluation behind it.
+
+## Known limitations
+
+- **The MTP (multi-token-prediction) block at layer 45 is excluded.** `transformers`' `Glm5NextForConditionalGeneration` does not instantiate it, so the pruning path cannot see it. Dropping it forecloses speculative decoding from this artifact.
+- REAP has no published data above 50% compression; this checkpoint sits at the validated
+  ceiling, not beyond it.
+- Expect **factual-recall** regression before reasoning or coding regression. That is the
+  measured failure mode of expert pruning on this architecture family: the closest published
+  analogue (`cerebras/Kimi-Linear-REAP-35B-A3B`, same KDA + full-attention stack) loses 3.4
+  points on FRAMES at only 30% pruning while code and maths hold flat.
+- Healing is an output-scale correction applied exactly to the F32 block scales, median gain **0.9090**. It was **measured**, not derived: post-prune routing is replayed from a cached router-score trace, because the first-moment estimator that pass 1 used ignores that `norm_topk_prob` renormalises the surviving top-8 and over-corrects by ~30%. It is *not* distillation and does not recover lost knowledge.
+- Routing is disrupted more than expert count suggests: the retained experts carry ~0.95x the
+  routing mass an average expert would, because REAP preserves rare-but-strong experts over
+  common-but-weak ones.
+
+## Serving on Jetson Thor
+
+Use the **cutlass** fused-MoE backend (the Marlin FP4 MoE kernel faults at >=256 experts) and
+`TRITON_MLA` for the 11 MLA+DSA layers (FLASHINFER is invalid for MLA).

pass3-hope/adapters/first_moment_gains.json

@@ -0,0 +1,61 @@
+{
+  "method": "first-moment MoE output-scale correction (no teacher, no forward pass)",
+  "gains": {
+    "model.language_model.layers.3.mlp": 0.6929448705738448,
+    "model.language_model.layers.4.mlp": 0.8275264171335417,
+    "model.language_model.layers.5.mlp": 0.7691773829390459,
+    "model.language_model.layers.6.mlp": 0.7979076218052154,
+    "model.language_model.layers.7.mlp": 0.8078911632468451,
+    "model.language_model.layers.8.mlp": 0.7168322158566274,
+    "model.language_model.layers.9.mlp": 0.7432604169992395,
+    "model.language_model.layers.10.mlp": 0.7206777935447581,
+    "model.language_model.layers.11.mlp": 0.7626100890374157,
+    "model.language_model.layers.12.mlp": 0.8074106247020925,
+    "model.language_model.layers.13.mlp": 0.7664205662606375,
+    "model.language_model.layers.14.mlp": 0.7833696012262404,
+    "model.language_model.layers.15.mlp": 0.7185073755181517,
+    "model.language_model.layers.16.mlp": 0.7727838850096236,
+    "model.language_model.layers.17.mlp": 0.7405453344473255,
+    "model.language_model.layers.18.mlp": 0.7578363968274796,
+    "model.language_model.layers.19.mlp": 0.7945049814830348,
+    "model.language_model.layers.20.mlp": 0.8107061170393675,
+    "model.language_model.layers.21.mlp": 0.8315814016946674,
+    "model.language_model.layers.22.mlp": 0.8532838478790715,
+    "model.language_model.layers.23.mlp": 0.7831526144395292,
+    "model.language_model.layers.24.mlp": 0.8134628560152266,
+    "model.language_model.layers.25.mlp": 0.7523038883232029,
+    "model.language_model.layers.26.mlp": 0.7902068124487682,
+    "model.language_model.layers.27.mlp": 0.7184067092556089,
+    "model.language_model.layers.28.mlp": 0.7484535209771637,
+    "model.language_model.layers.29.mlp": 0.7394112083006681,
+    "model.language_model.layers.30.mlp": 0.7531677997209874,
+    "model.language_model.layers.31.mlp": 0.7816384669181902,
+    "model.language_model.layers.32.mlp": 0.7665805247621434,
+    "model.language_model.layers.33.mlp": 0.733946518892489,
+    "model.language_model.layers.34.mlp": 0.7216211687255433,
+    "model.language_model.layers.35.mlp": 0.7269947802834427,
+    "model.language_model.layers.36.mlp": 0.7552838646710964,
+    "model.language_model.layers.37.mlp": 0.8013125057378521,
+    "model.language_model.layers.38.mlp": 0.7670837992974577,
+    "model.language_model.layers.39.mlp": 0.7986716776406351,
+    "model.language_model.layers.40.mlp": 0.7107730855763762,
+    "model.language_model.layers.41.mlp": 0.7380008004885258,
+    "model.language_model.layers.42.mlp": 0.7772684749907915,
+    "model.language_model.layers.43.mlp": 0.752188080793181,
+    "model.language_model.layers.44.mlp": 0.6673158203927475
+  },
+  "stats": {
+    "layers": 42,
+    "skipped": 0,
+    "gain_min": 0.6673,
+    "gain_max": 0.8533,
+    "gain_median": 0.7664,
+    "gain_mean": 0.7636
+  },
+  "per_expert_layers": [],
+  "per_expert_tensors": 0,
+  "per_expert_coefficients": {},
+  "experts_scaled": 6048,
+  "shards_rewritten": 59,
+  "not_done": "layer-local distillation and LoRA SFT - not tractable for 165B on a 117 GiB box; see wiki/70-healing.md"
+}

pass3-hope/chat_template.jinja

@@ -0,0 +1,261 @@
+[gMASK]<sop>
+{%- set effective_reasoning_effort = reasoning_effort if reasoning_effort is defined and reasoning_effort in ['low', 'high'] else 'max' -%}
+{%- if effective_reasoning_effort is not none -%}<|system|>Reasoning Effort: {{ effective_reasoning_effort | capitalize }}{%- endif -%}
+{%- set clear_thinking = clear_thinking if clear_thinking is defined else false -%}
+{%- if tools -%}
+{%- macro tool_to_json(tool) -%}
+    {%- set ns_tool = namespace(first=true) -%}
+    {{ '{' -}}
+    {%- for k, v in tool.items() -%}
+        {%- if k != 'defer_loading' and k != 'strict' -%}
+            {%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
+            {%- set ns_tool.first = false -%}
+            "{{ k }}": {{ v | tojson(ensure_ascii=False) }}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- '}' -}}
+{%- endmacro -%}
+{%- macro tool_references_to_response(refs) -%}
+    {{- '<tool_response><tools>\n' -}}
+    {%- for tr in refs -%}
+        {%- for tool in tools -%}
+            {%- if 'function' in tool -%}
+                {%- set tool = tool['function'] -%}
+            {%- endif -%}
+            {%- if tool.name == tr.name -%}
+                {{- tool_to_json(tool) + '\n' -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endfor -%}
+    {{- '</tools></tool_response>' -}}
+{%- endmacro -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{%- if 'function' in tool -%}
+    {%- set tool = tool['function'] -%}
+{%- endif -%}
+{% if tool.defer_loading is not defined or not tool.defer_loading %}
+{{ tool_to_json(tool) }}
+{% endif %}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
+{%- macro emit_image() -%}<|begin_of_image|><|image|><|end_of_image|>{%- endmacro -%}
+{%- macro emit_video() -%}<|begin_of_video|><|video|><|end_of_video|>{%- endmacro -%}
+{%- macro emit_audio() -%}<|begin_of_audio|><|end_of_audio|>{%- endmacro -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content -}}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text -}}
+            {%- elif item is string -%}
+                {{- item -}}
+            {%- elif item is mapping and item.type in ['image', 'image_url'] -%}
+                {{- emit_image() -}}
+            {%- elif item is mapping and item.type in ['video', 'video_url'] -%}
+                {{- emit_video() -}}
+            {%- elif item is mapping and item.type in ['audio', 'audio_url', 'input_audio'] -%}
+                {{- emit_audio() -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- elif content is not none -%}
+        {{- content -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro tool_response(text) -%}
+{{- '<tool_response>' + text + '</tool_response>' -}}
+{%- endmacro -%}
+{%- macro render_tool_response(m) -%}
+{%- if m.content is string -%}
+    {{- tool_response(m.content) -}}
+{%- elif m.content and m.content is not mapping and m.content.0.type == "tool_reference" -%}
+    {{- tool_references_to_response(m.content) -}}
+{%- elif is_list_of_outputs(m) -%}
+    {%- for tr in m.content -%}
+        {%- if tr.output is iterable and tr.output is not string and tr.output is not mapping and tr.output and tr.output.0.type == "tool_reference" -%}
+            {{- tool_references_to_response(tr.output) -}}
+        {%- else -%}
+            {{- tool_response(visible_text(tr.output)) -}}
+        {%- endif -%}
+    {%- endfor -%}
+{%- else -%}
+    {{- tool_response(visible_text(m.content)) -}}
+{%- endif -%}
+{%- endmacro -%}
+{%- macro id_of(obj) -%}
+    {%- if obj.tool_call_id -%}
+        {{- obj.tool_call_id -}}
+    {%- elif obj.id -%}
+        {{- obj.id -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro is_list_of_outputs(m) -%}
+    {%- if m.content and m.content.0.output is defined -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro has_dup_tool_result_id(lo, hi, target) -%}
+    {%- set ns_cnt = namespace(n=0) -%}
+    {%- for k in range(lo, hi + 1) -%}
+        {%- set m = messages[k] -%}
+        {%- if is_list_of_outputs(m) -%}
+            {%- for entry in m.content -%}
+                {%- if id_of(entry) == target -%}
+                    {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- elif id_of(m) == target -%}
+            {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+        {%- endif -%}
+        {%- if ns_cnt.n > 1 -%}{%- break -%}{%- endif -%}
+    {%- endfor -%}
+    {%- if ns_cnt.n > 1 -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro tc_id_exists(tcs, target) -%}
+    {%- set ns_f = namespace(found=false) -%}
+    {%- for tc in tcs -%}
+        {%- if id_of(tc) == target -%}
+            {%- set ns_f.found = true -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- if ns_f.found -%}1{%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) -%}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{%- for m in messages -%}
+{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- elif '</think>' in content %}
+    {%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
+    {%- set content = content.split('</think>')[-1] %}
+{%- endif %}
+{%- if (not clear_thinking or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
+{{ '<think>' + reasoning_content +  '</think>'}}
+{%- else -%}
+{{ '<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{- '<tool_call>' ~ tc.name -}}
+{% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' -}}
+    {%- set block_start = loop.index0 -%}
+    {%- set ns_blk = namespace(end=block_start) -%}
+    {%- for j in range(block_start, messages|length) -%}
+        {%- if messages[j].role == 'tool' -%}
+            {%- set ns_blk.end = j -%}
+        {%- else -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns_a = namespace(tool_calls=none) -%}
+    {%- if block_start > 0 and messages[block_start - 1].role == 'assistant' and messages[block_start - 1].tool_calls -%}
+        {%- set ns_a.tool_calls = messages[block_start - 1].tool_calls -%}
+    {%- endif -%}
+    {%- set ns_chk = namespace(can_sort=true) -%}
+    {%- if not ns_a.tool_calls -%}
+        {%- set ns_chk.can_sort = false -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {%- if not ns_chk.can_sort -%}{%- break -%}{%- endif -%}
+            {%- set m = messages[k] -%}
+            {%- if is_list_of_outputs(m) -%}
+                {%- for entry in m.content -%}
+                    {%- if not ns_chk.can_sort -%}{%- break -%}{%- endif -%}
+                    {%- set eid = id_of(entry) -%}
+                    {%- if not eid -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif has_dup_tool_result_id(block_start, ns_blk.end, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif not tc_id_exists(ns_a.tool_calls, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- else -%}
+                {%- set tk_id = id_of(m) -%}
+                {%- if not tk_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif has_dup_tool_result_id(block_start, ns_blk.end, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif not tc_id_exists(ns_a.tool_calls, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- for i in range(ns_a.tool_calls | length) -%}
+            {%- if not ns_chk.can_sort -%}{%- break -%}{%- endif -%}
+            {%- set tc_id = id_of(ns_a.tool_calls[i]) -%}
+            {%- if not tc_id -%}
+                {%- set ns_chk.can_sort = false -%}
+            {%- endif -%}
+            {%- for j in range(i + 1, ns_a.tool_calls | length) -%}
+                {%- if id_of(ns_a.tool_calls[j]) == tc_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                    {%- break -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- if ns_chk.can_sort -%}
+        {%- for tc in ns_a.tool_calls -%}
+            {%- set tc_id = id_of(tc) -%}
+            {%- for k in range(block_start, ns_blk.end + 1) -%}
+                {%- set m = messages[k] -%}
+                {%- if is_list_of_outputs(m) -%}
+                    {%- for entry in m.content -%}
+                        {%- set eid = id_of(entry) -%}
+                        {%- if eid == tc_id -%}
+                            {%- if entry.output is iterable and entry.output is not string and entry.output is not mapping and entry.output and entry.output.0.type == "tool_reference" -%}
+                                {{- tool_references_to_response(entry.output) -}}
+                            {%- else -%}
+                                {{- tool_response(visible_text(entry.output)) -}}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                {%- else -%}
+                    {%- set tk_id = id_of(m) -%}
+                    {%- if tk_id == tc_id -%}
+                        {{- render_tool_response(m) -}}
+                    {%- endif -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endfor -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {{- render_tool_response(messages[k]) -}}
+        {%- endfor -%}
+    {%- endif -%}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '<think>' -}}
+{%- endif -%}

pass3-hope/config.json

@@ -0,0 +1,1818 @@
+{
+  "architectures": [
+    "Glm5NextForConditionalGeneration"
+  ],
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "eos_token_id": [
+      154820,
+      154827,
+      154829
+    ],
+    "first_k_dense_replace": 3,
+    "hc_eps": 1e-06,
+    "hc_mult": 4,
+    "hc_sinkhorn_iters": 20,
+    "head_dim": 0,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "index_head_dim": 128,
+    "index_kpool": 4,
+    "index_kpool_always_select_tail": true,
+    "index_kpool_compress": true,
+    "index_n_heads": 32,
+    "index_topk": 2048,
+    "index_share_for_mtp_iteration": true,
+    "indexer_rope_interleave": true,
+    "indexer_types": [
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full"
+    ],
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "kv_lora_rank": 512,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention"
+    ],
+    "linear_attn_config": {
+      "num_heads": 64,
+      "gate_lower_bound": -5.0,
+      "head_dim": 128,
+      "short_conv_kernel_size": 4,
+      "kda_layers": [
+        0,
+        1,
+        2,
+        4,
+        5,
+        6,
+        8,
+        9,
+        10,
+        12,
+        13,
+        14,
+        16,
+        17,
+        18,
+        20,
+        21,
+        22,
+        24,
+        25,
+        26,
+        28,
+        29,
+        30,
+        32,
+        33,
+        34,
+        36,
+        37,
+        38,
+        40,
+        41,
+        42,
+        44
+      ],
+      "full_attn_layers": [
+        3,
+        7,
+        11,
+        15,
+        19,
+        23,
+        27,
+        31,
+        35,
+        39,
+        43
+      ]
+    },
+    "max_position_embeddings": 1048576,
+    "mhc": true,
+    "mla_use_nope": true,
+    "mlp_layer_types": [
+      "dense",
+      "dense",
+      "dense",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse"
+    ],
+    "model_type": "glm5_next_text",
+    "moe_intermediate_size": 2048,
+    "moe_router_dtype": "float32",
+    "n_group": 1,
+    "n_routed_experts": 144,
+    "n_shared_experts": 1,
+    "norm_topk_prob": true,
+    "num_attention_heads": 64,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 45,
+    "num_key_value_heads": 64,
+    "num_nextn_predict_layers": 1,
+    "output_router_logits": false,
+    "pad_token_id": 154820,
+    "q_lora_rank": 1536,
+    "qk_head_dim": 256,
+    "qk_nope_head_dim": 256,
+    "qk_rope_head_dim": 0,
+    "rms_norm_eps": 1e-05,
+    "routed_scaling_factor": 2.5,
+    "router_aux_loss_coef": 0.001,
+    "scoring_func": "sigmoid",
+    "swiglu_limit": 10.0,
+    "tie_word_embeddings": false,
+    "topk_group": 1,
+    "topk_method": "noaux_tc",
+    "use_cache": true,
+    "v_head_dim": 256,
+    "vocab_size": 154880,
+    "num_local_experts": 144
+  },
+  "vision_config": {
+    "attention_bias": true,
+    "attention_dropout": 0.0,
+    "depth": 24,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "image_size": 448,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "glm5_next_vision",
+    "num_heads": 16,
+    "out_hidden_size": 4096,
+    "patch_size": 14,
+    "projection_intermediate_size": 10240,
+    "rms_norm_eps": 1e-05,
+    "spatial_merge_size": 2,
+    "swiglu_limit": 10.0,
+    "temporal_patch_size": 2
+  },
+  "image_token_id": 154854,
+  "video_token_id": 154855,
+  "image_start_token_id": 154830,
+  "image_end_token_id": 154831,
+  "video_start_token_id": 154832,
+  "video_end_token_id": 154833,
+  "tie_word_embeddings": false,
+  "model_type": "glm5_next",
+  "transformers_version": "5.16.0",
+  "quantization_config": {
+    "activation_scheme": "dynamic",
+    "fmt": "e4m3",
+    "modules_to_not_convert": [
+      "attn_mha",
+      "attn_mqa",
+      "dt_bias",
+      "hyper_connection",
+      "lm_head",
+      "mapping_proj",
+      "model.embed_tokens",
+      "model.layers.0.hc_attn_base",
+      "model.layers.0.hc_attn_fn",
+      "model.layers.0.hc_attn_scale",
+      "model.layers.0.hc_ffn_base",
+      "model.layers.0.hc_ffn_fn",
+      "model.layers.0.hc_ffn_scale",
+      "model.layers.0.input_layernorm",
+      "model.layers.0.post_attention_layernorm",
+      "model.layers.0.self_attn.A_log",
+      "model.layers.0.self_attn.b_proj",
+      "model.layers.0.self_attn.dt_bias",
+      "model.layers.0.self_attn.f_a_proj",
+      "model.layers.0.self_attn.f_b_proj",
+      "model.layers.0.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.0.self_attn.g_a_proj",
+      "model.layers.0.self_attn.g_b_proj",
+      "model.layers.0.self_attn.k_conv1d",
+      "model.layers.0.self_attn.k_proj",
+      "model.layers.0.self_attn.o_norm",
+      "model.layers.0.self_attn.o_proj",
+      "model.layers.0.self_attn.q_conv1d",
+      "model.layers.0.self_attn.q_proj",
+      "model.layers.0.self_attn.qkv_proj",
+      "model.layers.0.self_attn.v_conv1d",
+      "model.layers.0.self_attn.v_proj",
+      "model.layers.1.hc_attn_base",
+      "model.layers.1.hc_attn_fn",
+      "model.layers.1.hc_attn_scale",
+      "model.layers.1.hc_ffn_base",
+      "model.layers.1.hc_ffn_fn",
+      "model.layers.1.hc_ffn_scale",
+      "model.layers.1.input_layernorm",
+      "model.layers.1.post_attention_layernorm",
+      "model.layers.1.self_attn.A_log",
+      "model.layers.1.self_attn.b_proj",
+      "model.layers.1.self_attn.dt_bias",
+      "model.layers.1.self_attn.f_a_proj",
+      "model.layers.1.self_attn.f_b_proj",
+      "model.layers.1.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.1.self_attn.g_a_proj",
+      "model.layers.1.self_attn.g_b_proj",
+      "model.layers.1.self_attn.k_conv1d",
+      "model.layers.1.self_attn.k_proj",
+      "model.layers.1.self_attn.o_norm",
+      "model.layers.1.self_attn.o_proj",
+      "model.layers.1.self_attn.q_conv1d",
+      "model.layers.1.self_attn.q_proj",
+      "model.layers.1.self_attn.qkv_proj",
+      "model.layers.1.self_attn.v_conv1d",
+      "model.layers.1.self_attn.v_proj",
+      "model.layers.10.hc_attn_base",
+      "model.layers.10.hc_attn_fn",
+      "model.layers.10.hc_attn_scale",
+      "model.layers.10.hc_ffn_base",
+      "model.layers.10.hc_ffn_fn",
+      "model.layers.10.hc_ffn_scale",
+      "model.layers.10.input_layernorm",
+      "model.layers.10.mlp.gate",
+      "model.layers.10.mlp.gate.e_score_correction_bias",
+      "model.layers.10.post_attention_layernorm",
+      "model.layers.10.self_attn.A_log",
+      "model.layers.10.self_attn.b_proj",
+      "model.layers.10.self_attn.dt_bias",
+      "model.layers.10.self_attn.f_a_proj",
+      "model.layers.10.self_attn.f_b_proj",
+      "model.layers.10.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.10.self_attn.g_a_proj",
+      "model.layers.10.self_attn.g_b_proj",
+      "model.layers.10.self_attn.k_conv1d",
+      "model.layers.10.self_attn.k_proj",
+      "model.layers.10.self_attn.o_norm",
+      "model.layers.10.self_attn.o_proj",
+      "model.layers.10.self_attn.q_conv1d",
+      "model.layers.10.self_attn.q_proj",
+      "model.layers.10.self_attn.qkv_proj",
+      "model.layers.10.self_attn.v_conv1d",
+      "model.layers.10.self_attn.v_proj",
+      "model.layers.11.hc_attn_base",
+      "model.layers.11.hc_attn_fn",
+      "model.layers.11.hc_attn_scale",
+      "model.layers.11.hc_ffn_base",
+      "model.layers.11.hc_ffn_fn",
+      "model.layers.11.hc_ffn_scale",
+      "model.layers.11.input_layernorm",
+      "model.layers.11.mlp.gate",
+      "model.layers.11.mlp.gate.e_score_correction_bias",
+      "model.layers.11.post_attention_layernorm",
+      "model.layers.11.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.11.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.11.self_attn.indexer.k_norm",
+      "model.layers.11.self_attn.indexer.k_norm.bias",
+      "model.layers.11.self_attn.indexer.weights_proj",
+      "model.layers.11.self_attn.indexer.wk",
+      "model.layers.11.self_attn.indexer.wq_b",
+      "model.layers.11.self_attn.kv_a_layernorm",
+      "model.layers.11.self_attn.kv_b_proj",
+      "model.layers.11.self_attn.q_a_layernorm",
+      "model.layers.12.hc_attn_base",
+      "model.layers.12.hc_attn_fn",
+      "model.layers.12.hc_attn_scale",
+      "model.layers.12.hc_ffn_base",
+      "model.layers.12.hc_ffn_fn",
+      "model.layers.12.hc_ffn_scale",
+      "model.layers.12.input_layernorm",
+      "model.layers.12.mlp.gate",
+      "model.layers.12.mlp.gate.e_score_correction_bias",
+      "model.layers.12.post_attention_layernorm",
+      "model.layers.12.self_attn.A_log",
+      "model.layers.12.self_attn.b_proj",
+      "model.layers.12.self_attn.dt_bias",
+      "model.layers.12.self_attn.f_a_proj",
+      "model.layers.12.self_attn.f_b_proj",
+      "model.layers.12.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.12.self_attn.g_a_proj",
+      "model.layers.12.self_attn.g_b_proj",
+      "model.layers.12.self_attn.k_conv1d",
+      "model.layers.12.self_attn.k_proj",
+      "model.layers.12.self_attn.o_norm",
+      "model.layers.12.self_attn.o_proj",
+      "model.layers.12.self_attn.q_conv1d",
+      "model.layers.12.self_attn.q_proj",
+      "model.layers.12.self_attn.qkv_proj",
+      "model.layers.12.self_attn.v_conv1d",
+      "model.layers.12.self_attn.v_proj",
+      "model.layers.13.hc_attn_base",
+      "model.layers.13.hc_attn_fn",
+      "model.layers.13.hc_attn_scale",
+      "model.layers.13.hc_ffn_base",
+      "model.layers.13.hc_ffn_fn",
+      "model.layers.13.hc_ffn_scale",
+      "model.layers.13.input_layernorm",
+      "model.layers.13.mlp.gate",
+      "model.layers.13.mlp.gate.e_score_correction_bias",
+      "model.layers.13.post_attention_layernorm",
+      "model.layers.13.self_attn.A_log",
+      "model.layers.13.self_attn.b_proj",
+      "model.layers.13.self_attn.dt_bias",
+      "model.layers.13.self_attn.f_a_proj",
+      "model.layers.13.self_attn.f_b_proj",
+      "model.layers.13.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.13.self_attn.g_a_proj",
+      "model.layers.13.self_attn.g_b_proj",
+      "model.layers.13.self_attn.k_conv1d",
+      "model.layers.13.self_attn.k_proj",
+      "model.layers.13.self_attn.o_norm",
+      "model.layers.13.self_attn.o_proj",
+      "model.layers.13.self_attn.q_conv1d",
+      "model.layers.13.self_attn.q_proj",
+      "model.layers.13.self_attn.qkv_proj",
+      "model.layers.13.self_attn.v_conv1d",
+      "model.layers.13.self_attn.v_proj",
+      "model.layers.14.hc_attn_base",
+      "model.layers.14.hc_attn_fn",
+      "model.layers.14.hc_attn_scale",
+      "model.layers.14.hc_ffn_base",
+      "model.layers.14.hc_ffn_fn",
+      "model.layers.14.hc_ffn_scale",
+      "model.layers.14.input_layernorm",
+      "model.layers.14.mlp.gate",
+      "model.layers.14.mlp.gate.e_score_correction_bias",
+      "model.layers.14.post_attention_layernorm",
+      "model.layers.14.self_attn.A_log",
+      "model.layers.14.self_attn.b_proj",
+      "model.layers.14.self_attn.dt_bias",
+      "model.layers.14.self_attn.f_a_proj",
+      "model.layers.14.self_attn.f_b_proj",
+      "model.layers.14.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.14.self_attn.g_a_proj",
+      "model.layers.14.self_attn.g_b_proj",
+      "model.layers.14.self_attn.k_conv1d",
+      "model.layers.14.self_attn.k_proj",
+      "model.layers.14.self_attn.o_norm",
+      "model.layers.14.self_attn.o_proj",
+      "model.layers.14.self_attn.q_conv1d",
+      "model.layers.14.self_attn.q_proj",
+      "model.layers.14.self_attn.qkv_proj",
+      "model.layers.14.self_attn.v_conv1d",
+      "model.layers.14.self_attn.v_proj",
+      "model.layers.15.hc_attn_base",
+      "model.layers.15.hc_attn_fn",
+      "model.layers.15.hc_attn_scale",
+      "model.layers.15.hc_ffn_base",
+      "model.layers.15.hc_ffn_fn",
+      "model.layers.15.hc_ffn_scale",
+      "model.layers.15.input_layernorm",
+      "model.layers.15.mlp.gate",
+      "model.layers.15.mlp.gate.e_score_correction_bias",
+      "model.layers.15.post_attention_layernorm",
+      "model.layers.15.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.15.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.15.self_attn.indexer.k_norm",
+      "model.layers.15.self_attn.indexer.k_norm.bias",
+      "model.layers.15.self_attn.indexer.weights_proj",
+      "model.layers.15.self_attn.indexer.wk",
+      "model.layers.15.self_attn.indexer.wq_b",
+      "model.layers.15.self_attn.kv_a_layernorm",
+      "model.layers.15.self_attn.kv_b_proj",
+      "model.layers.15.self_attn.q_a_layernorm",
+      "model.layers.16.hc_attn_base",
+      "model.layers.16.hc_attn_fn",
+      "model.layers.16.hc_attn_scale",
+      "model.layers.16.hc_ffn_base",
+      "model.layers.16.hc_ffn_fn",
+      "model.layers.16.hc_ffn_scale",
+      "model.layers.16.input_layernorm",
+      "model.layers.16.mlp.gate",
+      "model.layers.16.mlp.gate.e_score_correction_bias",
+      "model.layers.16.post_attention_layernorm",
+      "model.layers.16.self_attn.A_log",
+      "model.layers.16.self_attn.b_proj",
+      "model.layers.16.self_attn.dt_bias",
+      "model.layers.16.self_attn.f_a_proj",
+      "model.layers.16.self_attn.f_b_proj",
+      "model.layers.16.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.16.self_attn.g_a_proj",
+      "model.layers.16.self_attn.g_b_proj",
+      "model.layers.16.self_attn.k_conv1d",
+      "model.layers.16.self_attn.k_proj",
+      "model.layers.16.self_attn.o_norm",
+      "model.layers.16.self_attn.o_proj",
+      "model.layers.16.self_attn.q_conv1d",
+      "model.layers.16.self_attn.q_proj",
+      "model.layers.16.self_attn.qkv_proj",
+      "model.layers.16.self_attn.v_conv1d",
+      "model.layers.16.self_attn.v_proj",
+      "model.layers.17.hc_attn_base",
+      "model.layers.17.hc_attn_fn",
+      "model.layers.17.hc_attn_scale",
+      "model.layers.17.hc_ffn_base",
+      "model.layers.17.hc_ffn_fn",
+      "model.layers.17.hc_ffn_scale",
+      "model.layers.17.input_layernorm",
+      "model.layers.17.mlp.gate",
+      "model.layers.17.mlp.gate.e_score_correction_bias",
+      "model.layers.17.post_attention_layernorm",
+      "model.layers.17.self_attn.A_log",
+      "model.layers.17.self_attn.b_proj",
+      "model.layers.17.self_attn.dt_bias",
+      "model.layers.17.self_attn.f_a_proj",
+      "model.layers.17.self_attn.f_b_proj",
+      "model.layers.17.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.17.self_attn.g_a_proj",
+      "model.layers.17.self_attn.g_b_proj",
+      "model.layers.17.self_attn.k_conv1d",
+      "model.layers.17.self_attn.k_proj",
+      "model.layers.17.self_attn.o_norm",
+      "model.layers.17.self_attn.o_proj",
+      "model.layers.17.self_attn.q_conv1d",
+      "model.layers.17.self_attn.q_proj",
+      "model.layers.17.self_attn.qkv_proj",
+      "model.layers.17.self_attn.v_conv1d",
+      "model.layers.17.self_attn.v_proj",
+      "model.layers.18.hc_attn_base",
+      "model.layers.18.hc_attn_fn",
+      "model.layers.18.hc_attn_scale",
+      "model.layers.18.hc_ffn_base",
+      "model.layers.18.hc_ffn_fn",
+      "model.layers.18.hc_ffn_scale",
+      "model.layers.18.input_layernorm",
+      "model.layers.18.mlp.gate",
+      "model.layers.18.mlp.gate.e_score_correction_bias",
+      "model.layers.18.post_attention_layernorm",
+      "model.layers.18.self_attn.A_log",
+      "model.layers.18.self_attn.b_proj",
+      "model.layers.18.self_attn.dt_bias",
+      "model.layers.18.self_attn.f_a_proj",
+      "model.layers.18.self_attn.f_b_proj",
+      "model.layers.18.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.18.self_attn.g_a_proj",
+      "model.layers.18.self_attn.g_b_proj",
+      "model.layers.18.self_attn.k_conv1d",
+      "model.layers.18.self_attn.k_proj",
+      "model.layers.18.self_attn.o_norm",
+      "model.layers.18.self_attn.o_proj",
+      "model.layers.18.self_attn.q_conv1d",
+      "model.layers.18.self_attn.q_proj",
+      "model.layers.18.self_attn.qkv_proj",
+      "model.layers.18.self_attn.v_conv1d",
+      "model.layers.18.self_attn.v_proj",
+      "model.layers.19.hc_attn_base",
+      "model.layers.19.hc_attn_fn",
+      "model.layers.19.hc_attn_scale",
+      "model.layers.19.hc_ffn_base",
+      "model.layers.19.hc_ffn_fn",
+      "model.layers.19.hc_ffn_scale",
+      "model.layers.19.input_layernorm",
+      "model.layers.19.mlp.gate",
+      "model.layers.19.mlp.gate.e_score_correction_bias",
+      "model.layers.19.post_attention_layernorm",
+      "model.layers.19.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.19.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.19.self_attn.indexer.k_norm",
+      "model.layers.19.self_attn.indexer.k_norm.bias",
+      "model.layers.19.self_attn.indexer.weights_proj",
+      "model.layers.19.self_attn.indexer.wk",
+      "model.layers.19.self_attn.indexer.wq_b",
+      "model.layers.19.self_attn.kv_a_layernorm",
+      "model.layers.19.self_attn.kv_b_proj",
+      "model.layers.19.self_attn.q_a_layernorm",
+      "model.layers.2.hc_attn_base",
+      "model.layers.2.hc_attn_fn",
+      "model.layers.2.hc_attn_scale",
+      "model.layers.2.hc_ffn_base",
+      "model.layers.2.hc_ffn_fn",
+      "model.layers.2.hc_ffn_scale",
+      "model.layers.2.input_layernorm",
+      "model.layers.2.post_attention_layernorm",
+      "model.layers.2.self_attn.A_log",
+      "model.layers.2.self_attn.b_proj",
+      "model.layers.2.self_attn.dt_bias",
+      "model.layers.2.self_attn.f_a_proj",
+      "model.layers.2.self_attn.f_b_proj",
+      "model.layers.2.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.2.self_attn.g_a_proj",
+      "model.layers.2.self_attn.g_b_proj",
+      "model.layers.2.self_attn.k_conv1d",
+      "model.layers.2.self_attn.k_proj",
+      "model.layers.2.self_attn.o_norm",
+      "model.layers.2.self_attn.o_proj",
+      "model.layers.2.self_attn.q_conv1d",
+      "model.layers.2.self_attn.q_proj",
+      "model.layers.2.self_attn.qkv_proj",
+      "model.layers.2.self_attn.v_conv1d",
+      "model.layers.2.self_attn.v_proj",
+      "model.layers.20.hc_attn_base",
+      "model.layers.20.hc_attn_fn",
+      "model.layers.20.hc_attn_scale",
+      "model.layers.20.hc_ffn_base",
+      "model.layers.20.hc_ffn_fn",
+      "model.layers.20.hc_ffn_scale",
+      "model.layers.20.input_layernorm",
+      "model.layers.20.mlp.gate",
+      "model.layers.20.mlp.gate.e_score_correction_bias",
+      "model.layers.20.post_attention_layernorm",
+      "model.layers.20.self_attn.A_log",
+      "model.layers.20.self_attn.b_proj",
+      "model.layers.20.self_attn.dt_bias",
+      "model.layers.20.self_attn.f_a_proj",
+      "model.layers.20.self_attn.f_b_proj",
+      "model.layers.20.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.20.self_attn.g_a_proj",
+      "model.layers.20.self_attn.g_b_proj",
+      "model.layers.20.self_attn.k_conv1d",
+      "model.layers.20.self_attn.k_proj",
+      "model.layers.20.self_attn.o_norm",
+      "model.layers.20.self_attn.o_proj",
+      "model.layers.20.self_attn.q_conv1d",
+      "model.layers.20.self_attn.q_proj",
+      "model.layers.20.self_attn.qkv_proj",
+      "model.layers.20.self_attn.v_conv1d",
+      "model.layers.20.self_attn.v_proj",
+      "model.layers.21.hc_attn_base",
+      "model.layers.21.hc_attn_fn",
+      "model.layers.21.hc_attn_scale",
+      "model.layers.21.hc_ffn_base",
+      "model.layers.21.hc_ffn_fn",
+      "model.layers.21.hc_ffn_scale",
+      "model.layers.21.input_layernorm",
+      "model.layers.21.mlp.gate",
+      "model.layers.21.mlp.gate.e_score_correction_bias",
+      "model.layers.21.post_attention_layernorm",
+      "model.layers.21.self_attn.A_log",
+      "model.layers.21.self_attn.b_proj",
+      "model.layers.21.self_attn.dt_bias",
+      "model.layers.21.self_attn.f_a_proj",
+      "model.layers.21.self_attn.f_b_proj",
+      "model.layers.21.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.21.self_attn.g_a_proj",
+      "model.layers.21.self_attn.g_b_proj",
+      "model.layers.21.self_attn.k_conv1d",
+      "model.layers.21.self_attn.k_proj",
+      "model.layers.21.self_attn.o_norm",
+      "model.layers.21.self_attn.o_proj",
+      "model.layers.21.self_attn.q_conv1d",
+      "model.layers.21.self_attn.q_proj",
+      "model.layers.21.self_attn.qkv_proj",
+      "model.layers.21.self_attn.v_conv1d",
+      "model.layers.21.self_attn.v_proj",
+      "model.layers.22.hc_attn_base",
+      "model.layers.22.hc_attn_fn",
+      "model.layers.22.hc_attn_scale",
+      "model.layers.22.hc_ffn_base",
+      "model.layers.22.hc_ffn_fn",
+      "model.layers.22.hc_ffn_scale",
+      "model.layers.22.input_layernorm",
+      "model.layers.22.mlp.gate",
+      "model.layers.22.mlp.gate.e_score_correction_bias",
+      "model.layers.22.post_attention_layernorm",
+      "model.layers.22.self_attn.A_log",
+      "model.layers.22.self_attn.b_proj",
+      "model.layers.22.self_attn.dt_bias",
+      "model.layers.22.self_attn.f_a_proj",
+      "model.layers.22.self_attn.f_b_proj",
+      "model.layers.22.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.22.self_attn.g_a_proj",
+      "model.layers.22.self_attn.g_b_proj",
+      "model.layers.22.self_attn.k_conv1d",
+      "model.layers.22.self_attn.k_proj",
+      "model.layers.22.self_attn.o_norm",
+      "model.layers.22.self_attn.o_proj",
+      "model.layers.22.self_attn.q_conv1d",
+      "model.layers.22.self_attn.q_proj",
+      "model.layers.22.self_attn.qkv_proj",
+      "model.layers.22.self_attn.v_conv1d",
+      "model.layers.22.self_attn.v_proj",
+      "model.layers.23.hc_attn_base",
+      "model.layers.23.hc_attn_fn",
+      "model.layers.23.hc_attn_scale",
+      "model.layers.23.hc_ffn_base",
+      "model.layers.23.hc_ffn_fn",
+      "model.layers.23.hc_ffn_scale",
+      "model.layers.23.input_layernorm",
+      "model.layers.23.mlp.gate",
+      "model.layers.23.mlp.gate.e_score_correction_bias",
+      "model.layers.23.post_attention_layernorm",
+      "model.layers.23.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.23.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.23.self_attn.indexer.k_norm",
+      "model.layers.23.self_attn.indexer.k_norm.bias",
+      "model.layers.23.self_attn.indexer.weights_proj",
+      "model.layers.23.self_attn.indexer.wk",
+      "model.layers.23.self_attn.indexer.wq_b",
+      "model.layers.23.self_attn.kv_a_layernorm",
+      "model.layers.23.self_attn.kv_b_proj",
+      "model.layers.23.self_attn.q_a_layernorm",
+      "model.layers.24.hc_attn_base",
+      "model.layers.24.hc_attn_fn",
+      "model.layers.24.hc_attn_scale",
+      "model.layers.24.hc_ffn_base",
+      "model.layers.24.hc_ffn_fn",
+      "model.layers.24.hc_ffn_scale",
+      "model.layers.24.input_layernorm",
+      "model.layers.24.mlp.gate",
+      "model.layers.24.mlp.gate.e_score_correction_bias",
+      "model.layers.24.post_attention_layernorm",
+      "model.layers.24.self_attn.A_log",
+      "model.layers.24.self_attn.b_proj",
+      "model.layers.24.self_attn.dt_bias",
+      "model.layers.24.self_attn.f_a_proj",
+      "model.layers.24.self_attn.f_b_proj",
+      "model.layers.24.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.24.self_attn.g_a_proj",
+      "model.layers.24.self_attn.g_b_proj",
+      "model.layers.24.self_attn.k_conv1d",
+      "model.layers.24.self_attn.k_proj",
+      "model.layers.24.self_attn.o_norm",
+      "model.layers.24.self_attn.o_proj",
+      "model.layers.24.self_attn.q_conv1d",
+      "model.layers.24.self_attn.q_proj",
+      "model.layers.24.self_attn.qkv_proj",
+      "model.layers.24.self_attn.v_conv1d",
+      "model.layers.24.self_attn.v_proj",
+      "model.layers.25.hc_attn_base",
+      "model.layers.25.hc_attn_fn",
+      "model.layers.25.hc_attn_scale",
+      "model.layers.25.hc_ffn_base",
+      "model.layers.25.hc_ffn_fn",
+      "model.layers.25.hc_ffn_scale",
+      "model.layers.25.input_layernorm",
+      "model.layers.25.mlp.gate",
+      "model.layers.25.mlp.gate.e_score_correction_bias",
+      "model.layers.25.post_attention_layernorm",
+      "model.layers.25.self_attn.A_log",
+      "model.layers.25.self_attn.b_proj",
+      "model.layers.25.self_attn.dt_bias",
+      "model.layers.25.self_attn.f_a_proj",
+      "model.layers.25.self_attn.f_b_proj",
+      "model.layers.25.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.25.self_attn.g_a_proj",
+      "model.layers.25.self_attn.g_b_proj",
+      "model.layers.25.self_attn.k_conv1d",
+      "model.layers.25.self_attn.k_proj",
+      "model.layers.25.self_attn.o_norm",
+      "model.layers.25.self_attn.o_proj",
+      "model.layers.25.self_attn.q_conv1d",
+      "model.layers.25.self_attn.q_proj",
+      "model.layers.25.self_attn.qkv_proj",
+      "model.layers.25.self_attn.v_conv1d",
+      "model.layers.25.self_attn.v_proj",
+      "model.layers.26.hc_attn_base",
+      "model.layers.26.hc_attn_fn",
+      "model.layers.26.hc_attn_scale",
+      "model.layers.26.hc_ffn_base",
+      "model.layers.26.hc_ffn_fn",
+      "model.layers.26.hc_ffn_scale",
+      "model.layers.26.input_layernorm",
+      "model.layers.26.mlp.gate",
+      "model.layers.26.mlp.gate.e_score_correction_bias",
+      "model.layers.26.post_attention_layernorm",
+      "model.layers.26.self_attn.A_log",
+      "model.layers.26.self_attn.b_proj",
+      "model.layers.26.self_attn.dt_bias",
+      "model.layers.26.self_attn.f_a_proj",
+      "model.layers.26.self_attn.f_b_proj",
+      "model.layers.26.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.26.self_attn.g_a_proj",
+      "model.layers.26.self_attn.g_b_proj",
+      "model.layers.26.self_attn.k_conv1d",
+      "model.layers.26.self_attn.k_proj",
+      "model.layers.26.self_attn.o_norm",
+      "model.layers.26.self_attn.o_proj",
+      "model.layers.26.self_attn.q_conv1d",
+      "model.layers.26.self_attn.q_proj",
+      "model.layers.26.self_attn.qkv_proj",
+      "model.layers.26.self_attn.v_conv1d",
+      "model.layers.26.self_attn.v_proj",
+      "model.layers.27.hc_attn_base",
+      "model.layers.27.hc_attn_fn",
+      "model.layers.27.hc_attn_scale",
+      "model.layers.27.hc_ffn_base",
+      "model.layers.27.hc_ffn_fn",
+      "model.layers.27.hc_ffn_scale",
+      "model.layers.27.input_layernorm",
+      "model.layers.27.mlp.gate",
+      "model.layers.27.mlp.gate.e_score_correction_bias",
+      "model.layers.27.post_attention_layernorm",
+      "model.layers.27.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.27.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.27.self_attn.indexer.k_norm",
+      "model.layers.27.self_attn.indexer.k_norm.bias",
+      "model.layers.27.self_attn.indexer.weights_proj",
+      "model.layers.27.self_attn.indexer.wk",
+      "model.layers.27.self_attn.indexer.wq_b",
+      "model.layers.27.self_attn.kv_a_layernorm",
+      "model.layers.27.self_attn.kv_b_proj",
+      "model.layers.27.self_attn.q_a_layernorm",
+      "model.layers.28.hc_attn_base",
+      "model.layers.28.hc_attn_fn",
+      "model.layers.28.hc_attn_scale",
+      "model.layers.28.hc_ffn_base",
+      "model.layers.28.hc_ffn_fn",
+      "model.layers.28.hc_ffn_scale",
+      "model.layers.28.input_layernorm",
+      "model.layers.28.mlp.gate",
+      "model.layers.28.mlp.gate.e_score_correction_bias",
+      "model.layers.28.post_attention_layernorm",
+      "model.layers.28.self_attn.A_log",
+      "model.layers.28.self_attn.b_proj",
+      "model.layers.28.self_attn.dt_bias",
+      "model.layers.28.self_attn.f_a_proj",
+      "model.layers.28.self_attn.f_b_proj",
+      "model.layers.28.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.28.self_attn.g_a_proj",
+      "model.layers.28.self_attn.g_b_proj",
+      "model.layers.28.self_attn.k_conv1d",
+      "model.layers.28.self_attn.k_proj",
+      "model.layers.28.self_attn.o_norm",
+      "model.layers.28.self_attn.o_proj",
+      "model.layers.28.self_attn.q_conv1d",
+      "model.layers.28.self_attn.q_proj",
+      "model.layers.28.self_attn.qkv_proj",
+      "model.layers.28.self_attn.v_conv1d",
+      "model.layers.28.self_attn.v_proj",
+      "model.layers.29.hc_attn_base",
+      "model.layers.29.hc_attn_fn",
+      "model.layers.29.hc_attn_scale",
+      "model.layers.29.hc_ffn_base",
+      "model.layers.29.hc_ffn_fn",
+      "model.layers.29.hc_ffn_scale",
+      "model.layers.29.input_layernorm",
+      "model.layers.29.mlp.gate",
+      "model.layers.29.mlp.gate.e_score_correction_bias",
+      "model.layers.29.post_attention_layernorm",
+      "model.layers.29.self_attn.A_log",
+      "model.layers.29.self_attn.b_proj",
+      "model.layers.29.self_attn.dt_bias",
+      "model.layers.29.self_attn.f_a_proj",
+      "model.layers.29.self_attn.f_b_proj",
+      "model.layers.29.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.29.self_attn.g_a_proj",
+      "model.layers.29.self_attn.g_b_proj",
+      "model.layers.29.self_attn.k_conv1d",
+      "model.layers.29.self_attn.k_proj",
+      "model.layers.29.self_attn.o_norm",
+      "model.layers.29.self_attn.o_proj",
+      "model.layers.29.self_attn.q_conv1d",
+      "model.layers.29.self_attn.q_proj",
+      "model.layers.29.self_attn.qkv_proj",
+      "model.layers.29.self_attn.v_conv1d",
+      "model.layers.29.self_attn.v_proj",
+      "model.layers.3.hc_attn_base",
+      "model.layers.3.hc_attn_fn",
+      "model.layers.3.hc_attn_scale",
+      "model.layers.3.hc_ffn_base",
+      "model.layers.3.hc_ffn_fn",
+      "model.layers.3.hc_ffn_scale",
+      "model.layers.3.input_layernorm",
+      "model.layers.3.mlp.gate",
+      "model.layers.3.mlp.gate.e_score_correction_bias",
+      "model.layers.3.post_attention_layernorm",
+      "model.layers.3.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.3.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.3.self_attn.indexer.k_norm",
+      "model.layers.3.self_attn.indexer.k_norm.bias",
+      "model.layers.3.self_attn.indexer.weights_proj",
+      "model.layers.3.self_attn.indexer.wk",
+      "model.layers.3.self_attn.indexer.wq_b",
+      "model.layers.3.self_attn.kv_a_layernorm",
+      "model.layers.3.self_attn.kv_b_proj",
+      "model.layers.3.self_attn.q_a_layernorm",
+      "model.layers.30.hc_attn_base",
+      "model.layers.30.hc_attn_fn",
+      "model.layers.30.hc_attn_scale",
+      "model.layers.30.hc_ffn_base",
+      "model.layers.30.hc_ffn_fn",
+      "model.layers.30.hc_ffn_scale",
+      "model.layers.30.input_layernorm",
+      "model.layers.30.mlp.gate",
+      "model.layers.30.mlp.gate.e_score_correction_bias",
+      "model.layers.30.post_attention_layernorm",
+      "model.layers.30.self_attn.A_log",
+      "model.layers.30.self_attn.b_proj",
+      "model.layers.30.self_attn.dt_bias",
+      "model.layers.30.self_attn.f_a_proj",
+      "model.layers.30.self_attn.f_b_proj",
+      "model.layers.30.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.30.self_attn.g_a_proj",
+      "model.layers.30.self_attn.g_b_proj",
+      "model.layers.30.self_attn.k_conv1d",
+      "model.layers.30.self_attn.k_proj",
+      "model.layers.30.self_attn.o_norm",
+      "model.layers.30.self_attn.o_proj",
+      "model.layers.30.self_attn.q_conv1d",
+      "model.layers.30.self_attn.q_proj",
+      "model.layers.30.self_attn.qkv_proj",
+      "model.layers.30.self_attn.v_conv1d",
+      "model.layers.30.self_attn.v_proj",
+      "model.layers.31.hc_attn_base",
+      "model.layers.31.hc_attn_fn",
+      "model.layers.31.hc_attn_scale",
+      "model.layers.31.hc_ffn_base",
+      "model.layers.31.hc_ffn_fn",
+      "model.layers.31.hc_ffn_scale",
+      "model.layers.31.input_layernorm",
+      "model.layers.31.mlp.gate",
+      "model.layers.31.mlp.gate.e_score_correction_bias",
+      "model.layers.31.post_attention_layernorm",
+      "model.layers.31.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.31.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.31.self_attn.indexer.k_norm",
+      "model.layers.31.self_attn.indexer.k_norm.bias",
+      "model.layers.31.self_attn.indexer.weights_proj",
+      "model.layers.31.self_attn.indexer.wk",
+      "model.layers.31.self_attn.indexer.wq_b",
+      "model.layers.31.self_attn.kv_a_layernorm",
+      "model.layers.31.self_attn.kv_b_proj",
+      "model.layers.31.self_attn.q_a_layernorm",
+      "model.layers.32.hc_attn_base",
+      "model.layers.32.hc_attn_fn",
+      "model.layers.32.hc_attn_scale",
+      "model.layers.32.hc_ffn_base",
+      "model.layers.32.hc_ffn_fn",
+      "model.layers.32.hc_ffn_scale",
+      "model.layers.32.input_layernorm",
+      "model.layers.32.mlp.gate",
+      "model.layers.32.mlp.gate.e_score_correction_bias",
+      "model.layers.32.post_attention_layernorm",
+      "model.layers.32.self_attn.A_log",
+      "model.layers.32.self_attn.b_proj",
+      "model.layers.32.self_attn.dt_bias",
+      "model.layers.32.self_attn.f_a_proj",
+      "model.layers.32.self_attn.f_b_proj",
+      "model.layers.32.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.32.self_attn.g_a_proj",
+      "model.layers.32.self_attn.g_b_proj",
+      "model.layers.32.self_attn.k_conv1d",
+      "model.layers.32.self_attn.k_proj",
+      "model.layers.32.self_attn.o_norm",
+      "model.layers.32.self_attn.o_proj",
+      "model.layers.32.self_attn.q_conv1d",
+      "model.layers.32.self_attn.q_proj",
+      "model.layers.32.self_attn.qkv_proj",
+      "model.layers.32.self_attn.v_conv1d",
+      "model.layers.32.self_attn.v_proj",
+      "model.layers.33.hc_attn_base",
+      "model.layers.33.hc_attn_fn",
+      "model.layers.33.hc_attn_scale",
+      "model.layers.33.hc_ffn_base",
+      "model.layers.33.hc_ffn_fn",
+      "model.layers.33.hc_ffn_scale",
+      "model.layers.33.input_layernorm",
+      "model.layers.33.mlp.gate",
+      "model.layers.33.mlp.gate.e_score_correction_bias",
+      "model.layers.33.post_attention_layernorm",
+      "model.layers.33.self_attn.A_log",
+      "model.layers.33.self_attn.b_proj",
+      "model.layers.33.self_attn.dt_bias",
+      "model.layers.33.self_attn.f_a_proj",
+      "model.layers.33.self_attn.f_b_proj",
+      "model.layers.33.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.33.self_attn.g_a_proj",
+      "model.layers.33.self_attn.g_b_proj",
+      "model.layers.33.self_attn.k_conv1d",
+      "model.layers.33.self_attn.k_proj",
+      "model.layers.33.self_attn.o_norm",
+      "model.layers.33.self_attn.o_proj",
+      "model.layers.33.self_attn.q_conv1d",
+      "model.layers.33.self_attn.q_proj",
+      "model.layers.33.self_attn.qkv_proj",
+      "model.layers.33.self_attn.v_conv1d",
+      "model.layers.33.self_attn.v_proj",
+      "model.layers.34.hc_attn_base",
+      "model.layers.34.hc_attn_fn",
+      "model.layers.34.hc_attn_scale",
+      "model.layers.34.hc_ffn_base",
+      "model.layers.34.hc_ffn_fn",
+      "model.layers.34.hc_ffn_scale",
+      "model.layers.34.input_layernorm",
+      "model.layers.34.mlp.gate",
+      "model.layers.34.mlp.gate.e_score_correction_bias",
+      "model.layers.34.post_attention_layernorm",
+      "model.layers.34.self_attn.A_log",
+      "model.layers.34.self_attn.b_proj",
+      "model.layers.34.self_attn.dt_bias",
+      "model.layers.34.self_attn.f_a_proj",
+      "model.layers.34.self_attn.f_b_proj",
+      "model.layers.34.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.34.self_attn.g_a_proj",
+      "model.layers.34.self_attn.g_b_proj",
+      "model.layers.34.self_attn.k_conv1d",
+      "model.layers.34.self_attn.k_proj",
+      "model.layers.34.self_attn.o_norm",
+      "model.layers.34.self_attn.o_proj",
+      "model.layers.34.self_attn.q_conv1d",
+      "model.layers.34.self_attn.q_proj",
+      "model.layers.34.self_attn.qkv_proj",
+      "model.layers.34.self_attn.v_conv1d",
+      "model.layers.34.self_attn.v_proj",
+      "model.layers.35.hc_attn_base",
+      "model.layers.35.hc_attn_fn",
+      "model.layers.35.hc_attn_scale",
+      "model.layers.35.hc_ffn_base",
+      "model.layers.35.hc_ffn_fn",
+      "model.layers.35.hc_ffn_scale",
+      "model.layers.35.input_layernorm",
+      "model.layers.35.mlp.gate",
+      "model.layers.35.mlp.gate.e_score_correction_bias",
+      "model.layers.35.post_attention_layernorm",
+      "model.layers.35.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.35.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.35.self_attn.indexer.k_norm",
+      "model.layers.35.self_attn.indexer.k_norm.bias",
+      "model.layers.35.self_attn.indexer.weights_proj",
+      "model.layers.35.self_attn.indexer.wk",
+      "model.layers.35.self_attn.indexer.wq_b",
+      "model.layers.35.self_attn.kv_a_layernorm",
+      "model.layers.35.self_attn.kv_b_proj",
+      "model.layers.35.self_attn.q_a_layernorm",
+      "model.layers.36.hc_attn_base",
+      "model.layers.36.hc_attn_fn",
+      "model.layers.36.hc_attn_scale",
+      "model.layers.36.hc_ffn_base",
+      "model.layers.36.hc_ffn_fn",
+      "model.layers.36.hc_ffn_scale",
+      "model.layers.36.input_layernorm",
+      "model.layers.36.mlp.gate",
+      "model.layers.36.mlp.gate.e_score_correction_bias",
+      "model.layers.36.post_attention_layernorm",
+      "model.layers.36.self_attn.A_log",
+      "model.layers.36.self_attn.b_proj",
+      "model.layers.36.self_attn.dt_bias",
+      "model.layers.36.self_attn.f_a_proj",
+      "model.layers.36.self_attn.f_b_proj",
+      "model.layers.36.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.36.self_attn.g_a_proj",
+      "model.layers.36.self_attn.g_b_proj",
+      "model.layers.36.self_attn.k_conv1d",
+      "model.layers.36.self_attn.k_proj",
+      "model.layers.36.self_attn.o_norm",
+      "model.layers.36.self_attn.o_proj",
+      "model.layers.36.self_attn.q_conv1d",
+      "model.layers.36.self_attn.q_proj",
+      "model.layers.36.self_attn.qkv_proj",
+      "model.layers.36.self_attn.v_conv1d",
+      "model.layers.36.self_attn.v_proj",
+      "model.layers.37.hc_attn_base",
+      "model.layers.37.hc_attn_fn",
+      "model.layers.37.hc_attn_scale",
+      "model.layers.37.hc_ffn_base",
+      "model.layers.37.hc_ffn_fn",
+      "model.layers.37.hc_ffn_scale",
+      "model.layers.37.input_layernorm",
+      "model.layers.37.mlp.gate",
+      "model.layers.37.mlp.gate.e_score_correction_bias",
+      "model.layers.37.post_attention_layernorm",
+      "model.layers.37.self_attn.A_log",
+      "model.layers.37.self_attn.b_proj",
+      "model.layers.37.self_attn.dt_bias",
+      "model.layers.37.self_attn.f_a_proj",
+      "model.layers.37.self_attn.f_b_proj",
+      "model.layers.37.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.37.self_attn.g_a_proj",
+      "model.layers.37.self_attn.g_b_proj",
+      "model.layers.37.self_attn.k_conv1d",
+      "model.layers.37.self_attn.k_proj",
+      "model.layers.37.self_attn.o_norm",
+      "model.layers.37.self_attn.o_proj",
+      "model.layers.37.self_attn.q_conv1d",
+      "model.layers.37.self_attn.q_proj",
+      "model.layers.37.self_attn.qkv_proj",
+      "model.layers.37.self_attn.v_conv1d",
+      "model.layers.37.self_attn.v_proj",
+      "model.layers.38.hc_attn_base",
+      "model.layers.38.hc_attn_fn",
+      "model.layers.38.hc_attn_scale",
+      "model.layers.38.hc_ffn_base",
+      "model.layers.38.hc_ffn_fn",
+      "model.layers.38.hc_ffn_scale",
+      "model.layers.38.input_layernorm",
+      "model.layers.38.mlp.gate",
+      "model.layers.38.mlp.gate.e_score_correction_bias",
+      "model.layers.38.post_attention_layernorm",
+      "model.layers.38.self_attn.A_log",
+      "model.layers.38.self_attn.b_proj",
+      "model.layers.38.self_attn.dt_bias",
+      "model.layers.38.self_attn.f_a_proj",
+      "model.layers.38.self_attn.f_b_proj",
+      "model.layers.38.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.38.self_attn.g_a_proj",
+      "model.layers.38.self_attn.g_b_proj",
+      "model.layers.38.self_attn.k_conv1d",
+      "model.layers.38.self_attn.k_proj",
+      "model.layers.38.self_attn.o_norm",
+      "model.layers.38.self_attn.o_proj",
+      "model.layers.38.self_attn.q_conv1d",
+      "model.layers.38.self_attn.q_proj",
+      "model.layers.38.self_attn.qkv_proj",
+      "model.layers.38.self_attn.v_conv1d",
+      "model.layers.38.self_attn.v_proj",
+      "model.layers.39.hc_attn_base",
+      "model.layers.39.hc_attn_fn",
+      "model.layers.39.hc_attn_scale",
+      "model.layers.39.hc_ffn_base",
+      "model.layers.39.hc_ffn_fn",
+      "model.layers.39.hc_ffn_scale",
+      "model.layers.39.input_layernorm",
+      "model.layers.39.mlp.gate",
+      "model.layers.39.mlp.gate.e_score_correction_bias",
+      "model.layers.39.post_attention_layernorm",
+      "model.layers.39.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.39.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.39.self_attn.indexer.k_norm",
+      "model.layers.39.self_attn.indexer.k_norm.bias",
+      "model.layers.39.self_attn.indexer.weights_proj",
+      "model.layers.39.self_attn.indexer.wk",
+      "model.layers.39.self_attn.indexer.wq_b",
+      "model.layers.39.self_attn.kv_a_layernorm",
+      "model.layers.39.self_attn.kv_b_proj",
+      "model.layers.39.self_attn.q_a_layernorm",
+      "model.layers.4.hc_attn_base",
+      "model.layers.4.hc_attn_fn",
+      "model.layers.4.hc_attn_scale",
+      "model.layers.4.hc_ffn_base",
+      "model.layers.4.hc_ffn_fn",
+      "model.layers.4.hc_ffn_scale",
+      "model.layers.4.input_layernorm",
+      "model.layers.4.mlp.gate",
+      "model.layers.4.mlp.gate.e_score_correction_bias",
+      "model.layers.4.post_attention_layernorm",
+      "model.layers.4.self_attn.A_log",
+      "model.layers.4.self_attn.b_proj",
+      "model.layers.4.self_attn.dt_bias",
+      "model.layers.4.self_attn.f_a_proj",
+      "model.layers.4.self_attn.f_b_proj",
+      "model.layers.4.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.4.self_attn.g_a_proj",
+      "model.layers.4.self_attn.g_b_proj",
+      "model.layers.4.self_attn.k_conv1d",
+      "model.layers.4.self_attn.k_proj",
+      "model.layers.4.self_attn.o_norm",
+      "model.layers.4.self_attn.o_proj",
+      "model.layers.4.self_attn.q_conv1d",
+      "model.layers.4.self_attn.q_proj",
+      "model.layers.4.self_attn.qkv_proj",
+      "model.layers.4.self_attn.v_conv1d",
+      "model.layers.4.self_attn.v_proj",
+      "model.layers.40.hc_attn_base",
+      "model.layers.40.hc_attn_fn",
+      "model.layers.40.hc_attn_scale",
+      "model.layers.40.hc_ffn_base",
+      "model.layers.40.hc_ffn_fn",
+      "model.layers.40.hc_ffn_scale",
+      "model.layers.40.input_layernorm",
+      "model.layers.40.mlp.gate",
+      "model.layers.40.mlp.gate.e_score_correction_bias",
+      "model.layers.40.post_attention_layernorm",
+      "model.layers.40.self_attn.A_log",
+      "model.layers.40.self_attn.b_proj",
+      "model.layers.40.self_attn.dt_bias",
+      "model.layers.40.self_attn.f_a_proj",
+      "model.layers.40.self_attn.f_b_proj",
+      "model.layers.40.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.40.self_attn.g_a_proj",
+      "model.layers.40.self_attn.g_b_proj",
+      "model.layers.40.self_attn.k_conv1d",
+      "model.layers.40.self_attn.k_proj",
+      "model.layers.40.self_attn.o_norm",
+      "model.layers.40.self_attn.o_proj",
+      "model.layers.40.self_attn.q_conv1d",
+      "model.layers.40.self_attn.q_proj",
+      "model.layers.40.self_attn.qkv_proj",
+      "model.layers.40.self_attn.v_conv1d",
+      "model.layers.40.self_attn.v_proj",
+      "model.layers.41.hc_attn_base",
+      "model.layers.41.hc_attn_fn",
+      "model.layers.41.hc_attn_scale",
+      "model.layers.41.hc_ffn_base",
+      "model.layers.41.hc_ffn_fn",
+      "model.layers.41.hc_ffn_scale",
+      "model.layers.41.input_layernorm",
+      "model.layers.41.mlp.gate",
+      "model.layers.41.mlp.gate.e_score_correction_bias",
+      "model.layers.41.post_attention_layernorm",
+      "model.layers.41.self_attn.A_log",
+      "model.layers.41.self_attn.b_proj",
+      "model.layers.41.self_attn.dt_bias",
+      "model.layers.41.self_attn.f_a_proj",
+      "model.layers.41.self_attn.f_b_proj",
+      "model.layers.41.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.41.self_attn.g_a_proj",
+      "model.layers.41.self_attn.g_b_proj",
+      "model.layers.41.self_attn.k_conv1d",
+      "model.layers.41.self_attn.k_proj",
+      "model.layers.41.self_attn.o_norm",
+      "model.layers.41.self_attn.o_proj",
+      "model.layers.41.self_attn.q_conv1d",
+      "model.layers.41.self_attn.q_proj",
+      "model.layers.41.self_attn.qkv_proj",
+      "model.layers.41.self_attn.v_conv1d",
+      "model.layers.41.self_attn.v_proj",
+      "model.layers.42.hc_attn_base",
+      "model.layers.42.hc_attn_fn",
+      "model.layers.42.hc_attn_scale",
+      "model.layers.42.hc_ffn_base",
+      "model.layers.42.hc_ffn_fn",
+      "model.layers.42.hc_ffn_scale",
+      "model.layers.42.input_layernorm",
+      "model.layers.42.mlp.gate",
+      "model.layers.42.mlp.gate.e_score_correction_bias",
+      "model.layers.42.post_attention_layernorm",
+      "model.layers.42.self_attn.A_log",
+      "model.layers.42.self_attn.b_proj",
+      "model.layers.42.self_attn.dt_bias",
+      "model.layers.42.self_attn.f_a_proj",
+      "model.layers.42.self_attn.f_b_proj",
+      "model.layers.42.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.42.self_attn.g_a_proj",
+      "model.layers.42.self_attn.g_b_proj",
+      "model.layers.42.self_attn.k_conv1d",
+      "model.layers.42.self_attn.k_proj",
+      "model.layers.42.self_attn.o_norm",
+      "model.layers.42.self_attn.o_proj",
+      "model.layers.42.self_attn.q_conv1d",
+      "model.layers.42.self_attn.q_proj",
+      "model.layers.42.self_attn.qkv_proj",
+      "model.layers.42.self_attn.v_conv1d",
+      "model.layers.42.self_attn.v_proj",
+      "model.layers.43.hc_attn_base",
+      "model.layers.43.hc_attn_fn",
+      "model.layers.43.hc_attn_scale",
+      "model.layers.43.hc_ffn_base",
+      "model.layers.43.hc_ffn_fn",
+      "model.layers.43.hc_ffn_scale",
+      "model.layers.43.input_layernorm",
+      "model.layers.43.mlp.gate",
+      "model.layers.43.mlp.gate.e_score_correction_bias",
+      "model.layers.43.post_attention_layernorm",
+      "model.layers.43.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.43.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.43.self_attn.indexer.k_norm",
+      "model.layers.43.self_attn.indexer.k_norm.bias",
+      "model.layers.43.self_attn.indexer.weights_proj",
+      "model.layers.43.self_attn.indexer.wk",
+      "model.layers.43.self_attn.indexer.wq_b",
+      "model.layers.43.self_attn.kv_a_layernorm",
+      "model.layers.43.self_attn.kv_b_proj",
+      "model.layers.43.self_attn.q_a_layernorm",
+      "model.layers.44.hc_attn_base",
+      "model.layers.44.hc_attn_fn",
+      "model.layers.44.hc_attn_scale",
+      "model.layers.44.hc_ffn_base",
+      "model.layers.44.hc_ffn_fn",
+      "model.layers.44.hc_ffn_scale",
+      "model.layers.44.input_layernorm",
+      "model.layers.44.mlp.gate",
+      "model.layers.44.mlp.gate.e_score_correction_bias",
+      "model.layers.44.post_attention_layernorm",
+      "model.layers.44.self_attn.A_log",
+      "model.layers.44.self_attn.b_proj",
+      "model.layers.44.self_attn.dt_bias",
+      "model.layers.44.self_attn.f_a_proj",
+      "model.layers.44.self_attn.f_b_proj",
+      "model.layers.44.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.44.self_attn.g_a_proj",
+      "model.layers.44.self_attn.g_b_proj",
+      "model.layers.44.self_attn.k_conv1d",
+      "model.layers.44.self_attn.k_proj",
+      "model.layers.44.self_attn.o_norm",
+      "model.layers.44.self_attn.o_proj",
+      "model.layers.44.self_attn.q_conv1d",
+      "model.layers.44.self_attn.q_proj",
+      "model.layers.44.self_attn.qkv_proj",
+      "model.layers.44.self_attn.v_conv1d",
+      "model.layers.44.self_attn.v_proj",
+      "model.layers.45.eh_proj",
+      "model.layers.45.enorm",
+      "model.layers.45.hnorm",
+      "model.layers.45.input_layernorm",
+      "model.layers.45.mlp.gate",
+      "model.layers.45.mlp.gate.e_score_correction_bias",
+      "model.layers.45.post_attention_layernorm",
+      "model.layers.45.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.45.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.45.self_attn.indexer.k_norm",
+      "model.layers.45.self_attn.indexer.k_norm.bias",
+      "model.layers.45.self_attn.indexer.weights_proj",
+      "model.layers.45.self_attn.indexer.wk",
+      "model.layers.45.self_attn.indexer.wq_b",
+      "model.layers.45.self_attn.kv_a_layernorm",
+      "model.layers.45.self_attn.kv_b_proj",
+      "model.layers.45.self_attn.q_a_layernorm",
+      "model.layers.45.shared_head.norm",
+      "model.layers.5.hc_attn_base",
+      "model.layers.5.hc_attn_fn",
+      "model.layers.5.hc_attn_scale",
+      "model.layers.5.hc_ffn_base",
+      "model.layers.5.hc_ffn_fn",
+      "model.layers.5.hc_ffn_scale",
+      "model.layers.5.input_layernorm",
+      "model.layers.5.mlp.gate",
+      "model.layers.5.mlp.gate.e_score_correction_bias",
+      "model.layers.5.post_attention_layernorm",
+      "model.layers.5.self_attn.A_log",
+      "model.layers.5.self_attn.b_proj",
+      "model.layers.5.self_attn.dt_bias",
+      "model.layers.5.self_attn.f_a_proj",
+      "model.layers.5.self_attn.f_b_proj",
+      "model.layers.5.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.5.self_attn.g_a_proj",
+      "model.layers.5.self_attn.g_b_proj",
+      "model.layers.5.self_attn.k_conv1d",
+      "model.layers.5.self_attn.k_proj",
+      "model.layers.5.self_attn.o_norm",
+      "model.layers.5.self_attn.o_proj",
+      "model.layers.5.self_attn.q_conv1d",
+      "model.layers.5.self_attn.q_proj",
+      "model.layers.5.self_attn.qkv_proj",
+      "model.layers.5.self_attn.v_conv1d",
+      "model.layers.5.self_attn.v_proj",
+      "model.layers.6.hc_attn_base",
+      "model.layers.6.hc_attn_fn",
+      "model.layers.6.hc_attn_scale",
+      "model.layers.6.hc_ffn_base",
+      "model.layers.6.hc_ffn_fn",
+      "model.layers.6.hc_ffn_scale",
+      "model.layers.6.input_layernorm",
+      "model.layers.6.mlp.gate",
+      "model.layers.6.mlp.gate.e_score_correction_bias",
+      "model.layers.6.post_attention_layernorm",
+      "model.layers.6.self_attn.A_log",
+      "model.layers.6.self_attn.b_proj",
+      "model.layers.6.self_attn.dt_bias",
+      "model.layers.6.self_attn.f_a_proj",
+      "model.layers.6.self_attn.f_b_proj",
+      "model.layers.6.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.6.self_attn.g_a_proj",
+      "model.layers.6.self_attn.g_b_proj",
+      "model.layers.6.self_attn.k_conv1d",
+      "model.layers.6.self_attn.k_proj",
+      "model.layers.6.self_attn.o_norm",
+      "model.layers.6.self_attn.o_proj",
+      "model.layers.6.self_attn.q_conv1d",
+      "model.layers.6.self_attn.q_proj",
+      "model.layers.6.self_attn.qkv_proj",
+      "model.layers.6.self_attn.v_conv1d",
+      "model.layers.6.self_attn.v_proj",
+      "model.layers.7.hc_attn_base",
+      "model.layers.7.hc_attn_fn",
+      "model.layers.7.hc_attn_scale",
+      "model.layers.7.hc_ffn_base",
+      "model.layers.7.hc_ffn_fn",
+      "model.layers.7.hc_ffn_scale",
+      "model.layers.7.input_layernorm",
+      "model.layers.7.mlp.gate",
+      "model.layers.7.mlp.gate.e_score_correction_bias",
+      "model.layers.7.post_attention_layernorm",
+      "model.layers.7.self_attn.indexer.index_kpool_compress_ape",
+      "model.layers.7.self_attn.indexer.index_kpool_compress_gate",
+      "model.layers.7.self_attn.indexer.k_norm",
+      "model.layers.7.self_attn.indexer.k_norm.bias",
+      "model.layers.7.self_attn.indexer.weights_proj",
+      "model.layers.7.self_attn.indexer.wk",
+      "model.layers.7.self_attn.indexer.wq_b",
+      "model.layers.7.self_attn.kv_a_layernorm",
+      "model.layers.7.self_attn.kv_b_proj",
+      "model.layers.7.self_attn.q_a_layernorm",
+      "model.layers.8.hc_attn_base",
+      "model.layers.8.hc_attn_fn",
+      "model.layers.8.hc_attn_scale",
+      "model.layers.8.hc_ffn_base",
+      "model.layers.8.hc_ffn_fn",
+      "model.layers.8.hc_ffn_scale",
+      "model.layers.8.input_layernorm",
+      "model.layers.8.mlp.gate",
+      "model.layers.8.mlp.gate.e_score_correction_bias",
+      "model.layers.8.post_attention_layernorm",
+      "model.layers.8.self_attn.A_log",
+      "model.layers.8.self_attn.b_proj",
+      "model.layers.8.self_attn.dt_bias",
+      "model.layers.8.self_attn.f_a_proj",
+      "model.layers.8.self_attn.f_b_proj",
+      "model.layers.8.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.8.self_attn.g_a_proj",
+      "model.layers.8.self_attn.g_b_proj",
+      "model.layers.8.self_attn.k_conv1d",
+      "model.layers.8.self_attn.k_proj",
+      "model.layers.8.self_attn.o_norm",
+      "model.layers.8.self_attn.o_proj",
+      "model.layers.8.self_attn.q_conv1d",
+      "model.layers.8.self_attn.q_proj",
+      "model.layers.8.self_attn.qkv_proj",
+      "model.layers.8.self_attn.v_conv1d",
+      "model.layers.8.self_attn.v_proj",
+      "model.layers.9.hc_attn_base",
+      "model.layers.9.hc_attn_fn",
+      "model.layers.9.hc_attn_scale",
+      "model.layers.9.hc_ffn_base",
+      "model.layers.9.hc_ffn_fn",
+      "model.layers.9.hc_ffn_scale",
+      "model.layers.9.input_layernorm",
+      "model.layers.9.mlp.gate",
+      "model.layers.9.mlp.gate.e_score_correction_bias",
+      "model.layers.9.post_attention_layernorm",
+      "model.layers.9.self_attn.A_log",
+      "model.layers.9.self_attn.b_proj",
+      "model.layers.9.self_attn.dt_bias",
+      "model.layers.9.self_attn.f_a_proj",
+      "model.layers.9.self_attn.f_b_proj",
+      "model.layers.9.self_attn.fused_qkvbfg_a_proj",
+      "model.layers.9.self_attn.g_a_proj",
+      "model.layers.9.self_attn.g_b_proj",
+      "model.layers.9.self_attn.k_conv1d",
+      "model.layers.9.self_attn.k_proj",
+      "model.layers.9.self_attn.o_norm",
+      "model.layers.9.self_attn.o_proj",
+      "model.layers.9.self_attn.q_conv1d",
+      "model.layers.9.self_attn.q_proj",
+      "model.layers.9.self_attn.qkv_proj",
+      "model.layers.9.self_attn.v_conv1d",
+      "model.layers.9.self_attn.v_proj",
+      "model.norm",
+      "model.visual",
+      "router",
+      "visual",
+      "visual.blocks.0.attn.k_norm",
+      "visual.blocks.0.attn.proj",
+      "visual.blocks.0.attn.proj.bias",
+      "visual.blocks.0.attn.q_norm",
+      "visual.blocks.0.attn.qkv",
+      "visual.blocks.0.attn.qkv.bias",
+      "visual.blocks.0.mlp.down_proj",
+      "visual.blocks.0.mlp.down_proj.bias",
+      "visual.blocks.0.mlp.gate_proj",
+      "visual.blocks.0.mlp.gate_proj.bias",
+      "visual.blocks.0.mlp.up_proj",
+      "visual.blocks.0.mlp.up_proj.bias",
+      "visual.blocks.0.norm1",
+      "visual.blocks.0.norm2",
+      "visual.blocks.1.attn.k_norm",
+      "visual.blocks.1.attn.proj",
+      "visual.blocks.1.attn.proj.bias",
+      "visual.blocks.1.attn.q_norm",
+      "visual.blocks.1.attn.qkv",
+      "visual.blocks.1.attn.qkv.bias",
+      "visual.blocks.1.mlp.down_proj",
+      "visual.blocks.1.mlp.down_proj.bias",
+      "visual.blocks.1.mlp.gate_proj",
+      "visual.blocks.1.mlp.gate_proj.bias",
+      "visual.blocks.1.mlp.up_proj",
+      "visual.blocks.1.mlp.up_proj.bias",
+      "visual.blocks.1.norm1",
+      "visual.blocks.1.norm2",
+      "visual.blocks.10.attn.k_norm",
+      "visual.blocks.10.attn.proj",
+      "visual.blocks.10.attn.proj.bias",
+      "visual.blocks.10.attn.q_norm",
+      "visual.blocks.10.attn.qkv",
+      "visual.blocks.10.attn.qkv.bias",
+      "visual.blocks.10.mlp.down_proj",
+      "visual.blocks.10.mlp.down_proj.bias",
+      "visual.blocks.10.mlp.gate_proj",
+      "visual.blocks.10.mlp.gate_proj.bias",
+      "visual.blocks.10.mlp.up_proj",
+      "visual.blocks.10.mlp.up_proj.bias",
+      "visual.blocks.10.norm1",
+      "visual.blocks.10.norm2",
+      "visual.blocks.11.attn.k_norm",
+      "visual.blocks.11.attn.proj",
+      "visual.blocks.11.attn.proj.bias",
+      "visual.blocks.11.attn.q_norm",
+      "visual.blocks.11.attn.qkv",
+      "visual.blocks.11.attn.qkv.bias",
+      "visual.blocks.11.mlp.down_proj",
+      "visual.blocks.11.mlp.down_proj.bias",
+      "visual.blocks.11.mlp.gate_proj",
+      "visual.blocks.11.mlp.gate_proj.bias",
+      "visual.blocks.11.mlp.up_proj",
+      "visual.blocks.11.mlp.up_proj.bias",
+      "visual.blocks.11.norm1",
+      "visual.blocks.11.norm2",
+      "visual.blocks.12.attn.k_norm",
+      "visual.blocks.12.attn.proj",
+      "visual.blocks.12.attn.proj.bias",
+      "visual.blocks.12.attn.q_norm",
+      "visual.blocks.12.attn.qkv",
+      "visual.blocks.12.attn.qkv.bias",
+      "visual.blocks.12.mlp.down_proj",
+      "visual.blocks.12.mlp.down_proj.bias",
+      "visual.blocks.12.mlp.gate_proj",
+      "visual.blocks.12.mlp.gate_proj.bias",
+      "visual.blocks.12.mlp.up_proj",
+      "visual.blocks.12.mlp.up_proj.bias",
+      "visual.blocks.12.norm1",
+      "visual.blocks.12.norm2",
+      "visual.blocks.13.attn.k_norm",
+      "visual.blocks.13.attn.proj",
+      "visual.blocks.13.attn.proj.bias",
+      "visual.blocks.13.attn.q_norm",
+      "visual.blocks.13.attn.qkv",
+      "visual.blocks.13.attn.qkv.bias",
+      "visual.blocks.13.mlp.down_proj",
+      "visual.blocks.13.mlp.down_proj.bias",
+      "visual.blocks.13.mlp.gate_proj",
+      "visual.blocks.13.mlp.gate_proj.bias",
+      "visual.blocks.13.mlp.up_proj",
+      "visual.blocks.13.mlp.up_proj.bias",
+      "visual.blocks.13.norm1",
+      "visual.blocks.13.norm2",
+      "visual.blocks.14.attn.k_norm",
+      "visual.blocks.14.attn.proj",
+      "visual.blocks.14.attn.proj.bias",
+      "visual.blocks.14.attn.q_norm",
+      "visual.blocks.14.attn.qkv",
+      "visual.blocks.14.attn.qkv.bias",
+      "visual.blocks.14.mlp.down_proj",
+      "visual.blocks.14.mlp.down_proj.bias",
+      "visual.blocks.14.mlp.gate_proj",
+      "visual.blocks.14.mlp.gate_proj.bias",
+      "visual.blocks.14.mlp.up_proj",
+      "visual.blocks.14.mlp.up_proj.bias",
+      "visual.blocks.14.norm1",
+      "visual.blocks.14.norm2",
+      "visual.blocks.15.attn.k_norm",
+      "visual.blocks.15.attn.proj",
+      "visual.blocks.15.attn.proj.bias",
+      "visual.blocks.15.attn.q_norm",
+      "visual.blocks.15.attn.qkv",
+      "visual.blocks.15.attn.qkv.bias",
+      "visual.blocks.15.mlp.down_proj",
+      "visual.blocks.15.mlp.down_proj.bias",
+      "visual.blocks.15.mlp.gate_proj",
+      "visual.blocks.15.mlp.gate_proj.bias",
+      "visual.blocks.15.mlp.up_proj",
+      "visual.blocks.15.mlp.up_proj.bias",
+      "visual.blocks.15.norm1",
+      "visual.blocks.15.norm2",
+      "visual.blocks.16.attn.k_norm",
+      "visual.blocks.16.attn.proj",
+      "visual.blocks.16.attn.proj.bias",
+      "visual.blocks.16.attn.q_norm",
+      "visual.blocks.16.attn.qkv",
+      "visual.blocks.16.attn.qkv.bias",
+      "visual.blocks.16.mlp.down_proj",
+      "visual.blocks.16.mlp.down_proj.bias",
+      "visual.blocks.16.mlp.gate_proj",
+      "visual.blocks.16.mlp.gate_proj.bias",
+      "visual.blocks.16.mlp.up_proj",
+      "visual.blocks.16.mlp.up_proj.bias",
+      "visual.blocks.16.norm1",
+      "visual.blocks.16.norm2",
+      "visual.blocks.17.attn.k_norm",
+      "visual.blocks.17.attn.proj",
+      "visual.blocks.17.attn.proj.bias",
+      "visual.blocks.17.attn.q_norm",
+      "visual.blocks.17.attn.qkv",
+      "visual.blocks.17.attn.qkv.bias",
+      "visual.blocks.17.mlp.down_proj",
+      "visual.blocks.17.mlp.down_proj.bias",
+      "visual.blocks.17.mlp.gate_proj",
+      "visual.blocks.17.mlp.gate_proj.bias",
+      "visual.blocks.17.mlp.up_proj",
+      "visual.blocks.17.mlp.up_proj.bias",
+      "visual.blocks.17.norm1",
+      "visual.blocks.17.norm2",
+      "visual.blocks.18.attn.k_norm",
+      "visual.blocks.18.attn.proj",
+      "visual.blocks.18.attn.proj.bias",
+      "visual.blocks.18.attn.q_norm",
+      "visual.blocks.18.attn.qkv",
+      "visual.blocks.18.attn.qkv.bias",
+      "visual.blocks.18.mlp.down_proj",
+      "visual.blocks.18.mlp.down_proj.bias",
+      "visual.blocks.18.mlp.gate_proj",
+      "visual.blocks.18.mlp.gate_proj.bias",
+      "visual.blocks.18.mlp.up_proj",
+      "visual.blocks.18.mlp.up_proj.bias",
+      "visual.blocks.18.norm1",
+      "visual.blocks.18.norm2",
+      "visual.blocks.19.attn.k_norm",
+      "visual.blocks.19.attn.proj",
+      "visual.blocks.19.attn.proj.bias",
+      "visual.blocks.19.attn.q_norm",
+      "visual.blocks.19.attn.qkv",
+      "visual.blocks.19.attn.qkv.bias",
+      "visual.blocks.19.mlp.down_proj",
+      "visual.blocks.19.mlp.down_proj.bias",
+      "visual.blocks.19.mlp.gate_proj",
+      "visual.blocks.19.mlp.gate_proj.bias",
+      "visual.blocks.19.mlp.up_proj",
+      "visual.blocks.19.mlp.up_proj.bias",
+      "visual.blocks.19.norm1",
+      "visual.blocks.19.norm2",
+      "visual.blocks.2.attn.k_norm",
+      "visual.blocks.2.attn.proj",
+      "visual.blocks.2.attn.proj.bias",
+      "visual.blocks.2.attn.q_norm",
+      "visual.blocks.2.attn.qkv",
+      "visual.blocks.2.attn.qkv.bias",
+      "visual.blocks.2.mlp.down_proj",
+      "visual.blocks.2.mlp.down_proj.bias",
+      "visual.blocks.2.mlp.gate_proj",
+      "visual.blocks.2.mlp.gate_proj.bias",
+      "visual.blocks.2.mlp.up_proj",
+      "visual.blocks.2.mlp.up_proj.bias",
+      "visual.blocks.2.norm1",
+      "visual.blocks.2.norm2",
+      "visual.blocks.20.attn.k_norm",
+      "visual.blocks.20.attn.proj",
+      "visual.blocks.20.attn.proj.bias",
+      "visual.blocks.20.attn.q_norm",
+      "visual.blocks.20.attn.qkv",
+      "visual.blocks.20.attn.qkv.bias",
+      "visual.blocks.20.mlp.down_proj",
+      "visual.blocks.20.mlp.down_proj.bias",
+      "visual.blocks.20.mlp.gate_proj",
+      "visual.blocks.20.mlp.gate_proj.bias",
+      "visual.blocks.20.mlp.up_proj",
+      "visual.blocks.20.mlp.up_proj.bias",
+      "visual.blocks.20.norm1",
+      "visual.blocks.20.norm2",
+      "visual.blocks.21.attn.k_norm",
+      "visual.blocks.21.attn.proj",
+      "visual.blocks.21.attn.proj.bias",
+      "visual.blocks.21.attn.q_norm",
+      "visual.blocks.21.attn.qkv",
+      "visual.blocks.21.attn.qkv.bias",
+      "visual.blocks.21.mlp.down_proj",
+      "visual.blocks.21.mlp.down_proj.bias",
+      "visual.blocks.21.mlp.gate_proj",
+      "visual.blocks.21.mlp.gate_proj.bias",
+      "visual.blocks.21.mlp.up_proj",
+      "visual.blocks.21.mlp.up_proj.bias",
+      "visual.blocks.21.norm1",
+      "visual.blocks.21.norm2",
+      "visual.blocks.22.attn.k_norm",
+      "visual.blocks.22.attn.proj",
+      "visual.blocks.22.attn.proj.bias",
+      "visual.blocks.22.attn.q_norm",
+      "visual.blocks.22.attn.qkv",
+      "visual.blocks.22.attn.qkv.bias",
+      "visual.blocks.22.mlp.down_proj",
+      "visual.blocks.22.mlp.down_proj.bias",
+      "visual.blocks.22.mlp.gate_proj",
+      "visual.blocks.22.mlp.gate_proj.bias",
+      "visual.blocks.22.mlp.up_proj",
+      "visual.blocks.22.mlp.up_proj.bias",
+      "visual.blocks.22.norm1",
+      "visual.blocks.22.norm2",
+      "visual.blocks.23.attn.k_norm",
+      "visual.blocks.23.attn.proj",
+      "visual.blocks.23.attn.proj.bias",
+      "visual.blocks.23.attn.q_norm",
+      "visual.blocks.23.attn.qkv",
+      "visual.blocks.23.attn.qkv.bias",
+      "visual.blocks.23.mlp.down_proj",
+      "visual.blocks.23.mlp.down_proj.bias",
+      "visual.blocks.23.mlp.gate_proj",
+      "visual.blocks.23.mlp.gate_proj.bias",
+      "visual.blocks.23.mlp.up_proj",
+      "visual.blocks.23.mlp.up_proj.bias",
+      "visual.blocks.23.norm1",
+      "visual.blocks.23.norm2",
+      "visual.blocks.3.attn.k_norm",
+      "visual.blocks.3.attn.proj",
+      "visual.blocks.3.attn.proj.bias",
+      "visual.blocks.3.attn.q_norm",
+      "visual.blocks.3.attn.qkv",
+      "visual.blocks.3.attn.qkv.bias",
+      "visual.blocks.3.mlp.down_proj",
+      "visual.blocks.3.mlp.down_proj.bias",
+      "visual.blocks.3.mlp.gate_proj",
+      "visual.blocks.3.mlp.gate_proj.bias",
+      "visual.blocks.3.mlp.up_proj",
+      "visual.blocks.3.mlp.up_proj.bias",
+      "visual.blocks.3.norm1",
+      "visual.blocks.3.norm2",
+      "visual.blocks.4.attn.k_norm",
+      "visual.blocks.4.attn.proj",
+      "visual.blocks.4.attn.proj.bias",
+      "visual.blocks.4.attn.q_norm",
+      "visual.blocks.4.attn.qkv",
+      "visual.blocks.4.attn.qkv.bias",
+      "visual.blocks.4.mlp.down_proj",
+      "visual.blocks.4.mlp.down_proj.bias",
+      "visual.blocks.4.mlp.gate_proj",
+      "visual.blocks.4.mlp.gate_proj.bias",
+      "visual.blocks.4.mlp.up_proj",
+      "visual.blocks.4.mlp.up_proj.bias",
+      "visual.blocks.4.norm1",
+      "visual.blocks.4.norm2",
+      "visual.blocks.5.attn.k_norm",
+      "visual.blocks.5.attn.proj",
+      "visual.blocks.5.attn.proj.bias",
+      "visual.blocks.5.attn.q_norm",
+      "visual.blocks.5.attn.qkv",
+      "visual.blocks.5.attn.qkv.bias",
+      "visual.blocks.5.mlp.down_proj",
+      "visual.blocks.5.mlp.down_proj.bias",
+      "visual.blocks.5.mlp.gate_proj",
+      "visual.blocks.5.mlp.gate_proj.bias",
+      "visual.blocks.5.mlp.up_proj",
+      "visual.blocks.5.mlp.up_proj.bias",
+      "visual.blocks.5.norm1",
+      "visual.blocks.5.norm2",
+      "visual.blocks.6.attn.k_norm",
+      "visual.blocks.6.attn.proj",
+      "visual.blocks.6.attn.proj.bias",
+      "visual.blocks.6.attn.q_norm",
+      "visual.blocks.6.attn.qkv",
+      "visual.blocks.6.attn.qkv.bias",
+      "visual.blocks.6.mlp.down_proj",
+      "visual.blocks.6.mlp.down_proj.bias",
+      "visual.blocks.6.mlp.gate_proj",
+      "visual.blocks.6.mlp.gate_proj.bias",
+      "visual.blocks.6.mlp.up_proj",
+      "visual.blocks.6.mlp.up_proj.bias",
+      "visual.blocks.6.norm1",
+      "visual.blocks.6.norm2",
+      "visual.blocks.7.attn.k_norm",
+      "visual.blocks.7.attn.proj",
+      "visual.blocks.7.attn.proj.bias",
+      "visual.blocks.7.attn.q_norm",
+      "visual.blocks.7.attn.qkv",
+      "visual.blocks.7.attn.qkv.bias",
+      "visual.blocks.7.mlp.down_proj",
+      "visual.blocks.7.mlp.down_proj.bias",
+      "visual.blocks.7.mlp.gate_proj",
+      "visual.blocks.7.mlp.gate_proj.bias",
+      "visual.blocks.7.mlp.up_proj",
+      "visual.blocks.7.mlp.up_proj.bias",
+      "visual.blocks.7.norm1",
+      "visual.blocks.7.norm2",
+      "visual.blocks.8.attn.k_norm",
+      "visual.blocks.8.attn.proj",
+      "visual.blocks.8.attn.proj.bias",
+      "visual.blocks.8.attn.q_norm",
+      "visual.blocks.8.attn.qkv",
+      "visual.blocks.8.attn.qkv.bias",
+      "visual.blocks.8.mlp.down_proj",
+      "visual.blocks.8.mlp.down_proj.bias",
+      "visual.blocks.8.mlp.gate_proj",
+      "visual.blocks.8.mlp.gate_proj.bias",
+      "visual.blocks.8.mlp.up_proj",
+      "visual.blocks.8.mlp.up_proj.bias",
+      "visual.blocks.8.norm1",
+      "visual.blocks.8.norm2",
+      "visual.blocks.9.attn.k_norm",
+      "visual.blocks.9.attn.proj",
+      "visual.blocks.9.attn.proj.bias",
+      "visual.blocks.9.attn.q_norm",
+      "visual.blocks.9.attn.qkv",
+      "visual.blocks.9.attn.qkv.bias",
+      "visual.blocks.9.mlp.down_proj",
+      "visual.blocks.9.mlp.down_proj.bias",
+      "visual.blocks.9.mlp.gate_proj",
+      "visual.blocks.9.mlp.gate_proj.bias",
+      "visual.blocks.9.mlp.up_proj",
+      "visual.blocks.9.mlp.up_proj.bias",
+      "visual.blocks.9.norm1",
+      "visual.blocks.9.norm2",
+      "visual.downsample",
+      "visual.downsample.bias",
+      "visual.merger.down_proj",
+      "visual.merger.gate_proj",
+      "visual.merger.post_projection_norm",
+      "visual.merger.post_projection_norm.bias",
+      "visual.merger.proj",
+      "visual.merger.up_proj",
+      "visual.patch_embed.proj",
+      "visual.patch_embed.proj.bias",
+      "visual.post_layernorm",
+      "weights_proj"
+    ],
+    "quant_method": "fp8",
+    "weight_block_size": [
+      128,
+      128
+    ]
+  },
+  "reap": {
+    "ratio": 0.5,
+    "experts_kept": 144,
+    "experts_original": 288,
+    "mtp_preserved": true,
+    "mtp_criterion": "weight-norm (no activation saliency)"
+  }
+}

pass3-hope/generation_config.json

@@ -0,0 +1,12 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "pad_token_id": 154820,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "transformers_version": "5.16.0"
+}

pass3-hope/model-00001-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8039f2d791170802b82120cb527e7b1e7b3dde4ea39a8cfdcd3b0e9fb829a56b
+size 3989192520

pass3-hope/model-00002-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0874e1bbeef4abecafc148d1120dfe8977d780950c1794525f623008e9ae6a2
+size 3070696328

pass3-hope/model-00003-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e667e6063bfb83e3b636e5b05c8ef367b8b6c0bbab8b30c62aa1dee0c998b355
+size 2720321296

pass3-hope/model-00004-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1b284e254405e56e3d04b42458ada35db90fe07fbf06b0316979a892a9ba711
+size 2996917056

pass3-hope/model-00005-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ceb496ce7265e3d35789ffa4b7517a43c1fad499bf52411abc778476c67a122
+size 2644071536

pass3-hope/model-00006-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:835bff33d3eb7039313abd2e457d7f58bcec5228a967505685ec5b8388b9eac9
+size 2743836192

pass3-hope/model-00007-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d68a8a8f46a8b280cabffa0e902cbc171b1f171f64a6790b3aaf3085cea4f0d
+size 2820707520

pass3-hope/model-00008-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6759296aae2792cde5f9ceda646fa5017f1ba041201e88021107c2f06cef6090
+size 2669670640

pass3-hope/model-00009-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01ce7c5833cbfbba874cb222dd91deb1d4e25422769003b44a174c1970c93af2
+size 2820535080

pass3-hope/model-00010-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94146504c4faa375344c136c3bee31b533e862ab56247507fd25aa1441fd3cce
+size 2668490304

pass3-hope/model-00011-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d03172793a5f8af7bfd31fac27ee4e9258a0d7f085d523e45c55febcc1830f18
+size 2845453800

pass3-hope/model-00012-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:773e0f7fd3ee22a9db853cf5f0fba31acb09e108dd8d4add0b2932a7103a7a0e
+size 2896226056

pass3-hope/model-00013-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f1c496c055abe0b5e856830ca2e3f6d1030206a804b305bb0b25bfbd911edaa
+size 2693490560

pass3-hope/model-00014-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7798c0a472d0c8bdc6c29f580afe1d411b9b73b3ab5716c8074e3cc083a184ff
+size 2921398744

pass3-hope/model-00015-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9798b7301bcb9eb014714888a5d4855877c0bdaab979a304ef44d3b105dca65b
+size 2820707512

pass3-hope/model-00016-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4e8495140da8843cd70edb7d4726577120d0027c32cb9c0e7181818b139db43
+size 2617972144

pass3-hope/model-00017-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d94cc02477983e6d7f07f3fccab624b7522da8c99e0683eaf835967b0457b42d
+size 3072177576

pass3-hope/model-00018-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6007759c0715e7f594c34929c4fefb7ca4de029ea40de5e5f793129a5169218
+size 2669670600

pass3-hope/model-00019-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:879bc190d46d27c5b7f2131fc1eb882e33600de7373c689aafe291f02247c862
+size 2896226064

pass3-hope/model-00020-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a11a23cb68838ecf69ebcb0dc473ae1904800ffb5692083ae11d82375af230c0
+size 2643144944

pass3-hope/model-00021-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f242634082304f57993574fb0c949c0f47a32fab07325f4cdeade48d45650a01
+size 3022089912

pass3-hope/model-00022-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce73f03fe1628a915b836780edf47f7ef00745b07c3d32758f7f5d002460b7e6
+size 2618898680

pass3-hope/model-00023-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea1646fc9d95266ed874af4ec5cd439497b2bb68b8b131cada73ce8dfba4dc2c
+size 2718663384

pass3-hope/model-00024-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05fd22bcd3b04067073e9ef97cd2b3cf5c0397f185bed5f0ecacd844b33e5241
+size 2795534776

pass3-hope/model-00025-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bca74234f05519870a8b0620fdb98470f6c2b8853bfbada4a41171c5ce6d6260
+size 2745189016

pass3-hope/model-00026-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1af8ca92e6cc327bf9d42529e98852a2963055d7d25cc41acfe00714690387c3
+size 2971744504

pass3-hope/model-00027-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:102915ca4b454d3494e4b4f3d9229729342264f94a1989fe9d460b1997112dbe
+size 2668317768

pass3-hope/model-00028-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:119057b3307e76960aa9dea81b6dad90d2cb9e9653d8fc00877cd1b94e4fec7a
+size 2719589800

pass3-hope/model-00029-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:390f0e01b6d53c3ab5e9644d946ff82587daa29bef84fa81ed45a432b1a63f5b
+size 2845880360

pass3-hope/model-00030-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8b3a983dd781d48abc4cd1dcf0f42346031389e6845cfdbe4206d2db9e792d2
+size 2693490576

pass3-hope/model-00031-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6855529abddedeb1bd8624cecb89846c7d740a74554c64d233961fd0dd0213b7
+size 2745188640

pass3-hope/model-00032-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6966e4b2d6ce88d147d2780ea4a958737b7f23e43a86e288d286b53279f18ec5
+size 2996490296

pass3-hope/model-00033-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63fdb671316daa61335fa97d75555225f63404d336bc93a21f47928e74e6abf5
+size 2720016416

pass3-hope/model-00034-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4b29c6cadabc80352f2ace69debd3a148b8a2b04ce08b05b0febe853941c2c8
+size 2743836192

pass3-hope/model-00035-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:628e7f8cff55a6618b2e148225edb9ce17133e18847ecff28d3c81e6cda803ac
+size 2920972280

pass3-hope/model-00036-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca678499b2852941b10c637f664e7011942327c6a674e2702500da3c9c6f270b
+size 2594152280

pass3-hope/model-00037-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e624bddd389ab3482e7712f2fcd5c4a8c441ee36e7734f5d0416a10e29c270fe
+size 2769008992

pass3-hope/model-00038-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:483d70dd699c63762704f77096d42b782da9a3ba1e407164d42f7539ac693a82
+size 2745189168

pass3-hope/model-00039-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4dfbcdbcf75fd2d72fad6b60f6ebb4c86c568f6edb213c86225394aa62f55adf
+size 3022089904

pass3-hope/model-00040-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc7558e341126f73b0abf4835937ea5adb19efb5b2ce14cdbb7c5ca58d22f206
+size 2644071552

pass3-hope/model-00041-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b19d0077f3c285da2d73fbccae29613a2a0167ce5fbc92827223ff1203eac6a
+size 2542453712

pass3-hope/model-00042-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5edd53fc9fe1670d249ee0970b15e264ff7555dcd55969092ecc77ef76475950
+size 2896225896

pass3-hope/model-00043-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efa61a9b9ad7ad743287c5308b2d810c278708447a43977df94a7b72ee733a23
+size 2820707544

pass3-hope/model-00044-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6094e25a56cc70d9022e7d09b9f115725a46ca87b99a30a3e79362a46909928
+size 2693490576