Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,13 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+media/SWE-rebench-V2-pipeline.png filter=lfs diff=lfs merge=lfs -text
+media/glm47flash_throughput.png filter=lfs diff=lfs merge=lfs -text
+media/learned_patterns.png filter=lfs diff=lfs merge=lfs -text
+media/overlay_sandbox.png filter=lfs diff=lfs merge=lfs -text
+media/reasoning-tool-calls-logprobs.png filter=lfs diff=lfs merge=lfs -text
+media/score_vs_cost.png filter=lfs diff=lfs merge=lfs -text
+media/trace_overview.png filter=lfs diff=lfs merge=lfs -text
+media/traffic_recorder.png filter=lfs diff=lfs merge=lfs -text
+media/train_loss_decomposition.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,408 @@
 ---
+base_model:
+- whitecircle/GLM-4.7-Flash-Coder
 license: apache-2.0
+pipeline_tag: text-to-text
+library_name: transformers
+tags:
+- moe
+- mixture-of-experts
+- glm
+- glm-4.7-flash
+- code
+- coder
+- nvfp4
 ---
+# Model Card
+
+
+
+Source Model: [whitecircle/GLM-4.7-Flash-Coder](https://huggingface.co/whitecircle/GLM-4.7-Flash-Coder)
+
+
+
+
+<div style="max-width: 70%; margin: 0 auto; text-align: center; font-weight: bold;">
+  THIS REPO:
+</div>
+<div style="max-width: 70%; margin: 0 auto; text-align: center; font-weight: bold;">
+ NVFP4 quantization for use on Blackwell gpus sm100 sm120 sm121.
+</div>
+
+
+Ready to serve on latest vLLM branch. 
+
+Please take note of the server configuration below.
+
+
+
+
+Quantization via NVIDIA/Model-Optimizer custom workflow:
+| Component | Precision |
+|---|---|
+| Routed experts | NVFP4 W4A4 (group 16, FP8 block scales, FP32 global scales) |
+| Shared expert, router, attention, embeddings, `lm_head` | BF16 |
+| MTP | no MTP in this repo |
+| KV cache | Not quantized |
+Please see post-quantization [verification_report](https://huggingface.co/cleyesode/GLM-4.7-Flash-Coder-NVFP4/verification_report.txt) for details. 
+
+
+
+Calibrated on curated samples from [Nickyang/RazorCal](https://huggingface.co/datasets/Nickyang/RazorCal):
+
+
+| Domain                    | Count   | Percentage | Description / Sub-sources                                                           |
+| :------------------------ | :------ | :--------- | :---------------------------------------------------------------------------------- |
+| **Coding**                | 57      | \~24.1%    | Algorithmic Python/C++, competitive programming, and GitHub SWE issue resolution.   |
+| **Math**                  | 41      | \~17.3%    | High-level math derivations, AIME-style problems, calculus, and geometry.           |
+| **Chinese-STEM**          | 33      | \~13.9%    | Multilingual math, coding, and science MCQs written in Chinese.                     |
+| **Instruction Following** | 32      | \~13.5%    | Strict formatting constraints (e.g., word count, excluded words, required letters). |
+| **World Knowledge**       | 30      | \~12.7%    | Trivia, history, reading comprehension, and translation tasks.                      |
+| **Tool Calling**          | 25      | \~10.5%    | Multi-turn agentic workflows using external APIs (weather, stocks, ecommerce).      |
+| **Science (STEM)**        | 19      | \~8.0%     | Chemistry, physics, biology, and thermodynamics (both MCQs and long-form).          |
+
+
+
+
+## Serving with vLLM
+
+Example: 4 x 5060Ti 16GB | tensor parallel 4 with expert parallelism | single sequence at 175k context
+
+**Note:** kv-cache-dtype quants not well supported on this architecture in vllm - BF16 kv config shown here
+
+```bash
+export CUDA_DEVICE_ORDER=PCI_BUS_ID
+export CUDA_VISIBLE_DEVICES=0,1,2,3
+export PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True"
+export VLLM_USE_RUST_FRONTEND=0
+
+
+vllm serve cleyesode/GLM-4.7-Flash-Coder-NVFP4 \
+    --served-model-name GLM-4.7-Flash-Coder-NVFP4 \
+    --host 0.0.0.0 \
+    --port 8000 \
+    --distributed-executor-backend mp \
+    --tensor-parallel-size 4 \
+    --enable-expert-parallel \
+    --kv-cache-memory 9567537324 \
+    --max-model-len 175000 \
+    --max-num-seqs 1 \
+    --max-num-batched-tokens 1024 \
+    --enable-prefix-caching \
+    --enable-chunked-prefill \
+    --tool-call-parser glm47 \
+    --reasoning-parser glm45 \
+    --enable-auto-tool-choice \
+    --attention-backend triton_mla \
+    --moe-backend flashinfer_cutlass \
+    --kv-cache-dtype bfloat16 \
+    --no-enable-flashinfer-autotune \
+    --trust-remote-code
+```
+
+
+## Memory Requirements
+
+  - VRAM: ~63GiB (per the above tp4 example)
+
+
+## Metrics (compute-only)
+
+Very impressive generation speed absent MTP.
+
+Throughput metrics collected on **real prompts**:
+
+
+| Input tokens | Output tokens | TTFT (ms) | TTFT p99 (ms) | Prefill (tok/s) | TPOT (ms) | ITL p99 (ms) | **Decode (tok/s)** | E2E latency (s) | Draft acceptance | Mean accepted length |
+|-------------:|--------------:|----------:|--------------:|----------------:|----------:|-------------:|---------------:|----------------:|-----------------:|---------------------:|
+|          956 |           512 |       104 |           135 |           9,158 |       9.0 |          9.8 |          **111.0** |             4.7 |               -- |                   -- |
+|        8,127 |           512 |     1,085 |         1,094 |           7,493 |      10.3 |         11.1 |           **96.9** |             6.4 |               -- |                   -- |
+|       32,705 |           512 |     8,759 |         8,769 |           3,734 |      14.4 |         15.0 |           **69.3** |            16.1 |               -- |                   -- |
+|       65,451 |           512 |    29,482 |        29,484 |           2,220 |      19.8 |         20.5 |           **50.4** |            39.5 |               -- |                   -- |
+|      126,975 |           512 |   101,749 |       101,809 |           1,248 |      28.9 |         29.7 |           **34.6** |           116.6 |               -- |                   -- |
+
+## Benchmarks
+
+  **No performance benchmarks completed for NVFP4 model at this time**
+
+
+
+
+---
+
+Source Repo Model Card:
+
+# Introducing ⚡ GLM-4.7-Flash-Coder ⚡
+
+We fine-tuned GLM-4.7-Flash to push its agentic coding capabilities further: from 33.15% to 41.65% on SWE-rebench-V2 (+26% relative improvement) at 14% lower inference cost.
+
+This 30B Mixture-of-Experts model beats competitors 3.5-4× its size — Qwen3.5-122B-A10B and Devstral-2 — and surpasses Ling-2.6-Flash by a narrow margin.
+
+We trained it with [Halo](https://github.com/whitecircle/halo), the post-training framework we’re open-sourcing today — the full story of the framework [here](https://whitecircle.com/research).
+
+![Score vs cost-per-task scatter plot on SWE-rebench-V2, 500 Python tasks: GLM-4.7-Flash-Coder at 0.417 sits 26% above GLM-4.7-Flash at 0.332, at a lower cost per task](media/score_vs_cost.png)
+
+## More reasoning, fewer turns
+
+Although GLM-4.7-Flash-Coder does 2.4× more reasoning than the base model, it completes tasks faster: 45% fewer turns, which translates to 14% lower inference cost.
+
+![Box plots for base vs SFT model: median reasoning tokens per response 63 vs 150, assistant turns per task 82.5 vs 44, trace token length 52k vs 39k, average cost per task $0.0443 vs $0.0382](media/trace_overview.png)
+
+## The key to turn efficiency is concurrent tool calls
+
+The fine-tuned model learned to batch its tool use instead of working one call at a time. Turns with two or more parallel `read` calls jumped from 1.6% to 9.7% of all turns, parallel `bash` from 0.6% to 7.2%, and parallel `edit`, which is almost nonexistent in the base model, grew ~90×. Reading three files in one turn instead of three turns is exactly the kind of habit that compounds into 45% fewer turns overall.
+
+![Frequency of responses with 2+ read, bash, and edit tool calls: the SFT model issues simultaneous tool calls 6–90× more often than the base model](media/simultaneous_tool_calls.png)
+
+## SWE-rebench-V2
+
+We trained and evaluated models on [nebius/SWE-rebench-V2](https://huggingface.co/datasets/nebius/SWE-rebench-V2), a dataset of 32k agentic coding tasks. It consists of real issues from public repositories, closely representing real-world SWE challenges.
+
+![SWE-rebench-V2 pipeline: a repository and an issue enter an agentic loop in a sandbox, the generated patch is checked by the test-suite verifier, and the reward is 1 if all tests pass](media/SWE-rebench-V2-pipeline.png)
+
+## Teacher — GLM-5.1
+
+The training data comes from GLM-5.1 acting as a teacher. We ran it on the train split with 4 rollouts per task and kept only trajectories that passed the tests: 7,777 successful multi-turn traces after rejection sampling.
+
+We’re releasing the [training dataset](https://huggingface.co/datasets/whitecircle/swe-rebench-v2-glm-5.1-pi-agent-successful-traces) alongside the [train/test splits](https://huggingface.co/datasets/whitecircle/swe-rebench-v2-clean-python-tasks).
+
+All models run inside PI — a lightweight agent harness with four tools: `bash`, `read`, `edit`, `write`.
+
+## Training
+
+We trained Coder with Supervised Fine-Tuning (SFT) using [Halo](https://github.com/whitecircle/halo), our post-training framework we’re open-sourcing today 🎉
+
+- Parallelism mode: DP=16 (multi-node training)
+- Context length: 75k
+- Packing: Enabled
+- Global batch: 16
+- Learning rate scheduling: Cosine, with linear warm-up
+- Training time: 3 hours on 16 B300 GPUs
+
+### Halo 😇 is driven entirely by a single YAML config
+
+```yaml
+model_name_or_path: zai-org/GLM-4.7-Flash
+trust_remote_code: true
+attn_implementation: flash_attention_4_hybrid
+
+dataset:
+- whitecircle/swe-rebench-v2-glm-5.1-pi-agent-successful-traces
+test_size: 0.01
+
+conversation_field: messages
+tools_field: tools
+interleaved_thinking: true
+
+max_length: 75000
+packing: true
+
+learning_rate: 1e-5
+lr_scheduler_type: cosine
+warmup_steps: 25
+per_device_train_batch_size: 1
+per_device_eval_batch_size: 1
+
+moe_balancing: none
+liger_kernel_config:
+  fused_linear_cross_entropy: true
+
+assistant_message_template: "<|assistant|>"
+
+output_dir: /mnt/research/GLM-4.7-Flash-Coder
+save_steps: 50
+save_only_model: true
+
+eval_strategy: steps
+eval_steps: 50
+logging_steps: 1
+logging_first_step: true
+log_decoded_samples: true
+
+report_to: wandb
+project_name: agentic-sft
+run_name: glm-4.7-flash-coder
+
+enable_efficiency_metrics: true
+```
+
+## Emergent skills
+
+The model picked up several of the teacher’s skills and patterns during fine-tuning:
+
+- A 10× higher rate of self-corrections in reasoning,
+- Self-directed questions (“But wait, could this ever be True?”),
+- Advanced tool calling patterns, e.g. setting timeout for the `bash` tool.
+
+![Bar charts of self-corrections, question marks in reasoning, git stash usage, and timeouts passed to bash across base, SFT, and teacher models](media/learned_patterns.png)
+
+Perhaps the most surprising transfer was the `git stash` usage:
+
+> *Hmm, actually these look like pre-existing test differences unrelated to my fix. Let me revert my fix and run the test again.*
+>
+> ```bash
+> git stash && python -m pytest tests/test_markdown.py::test_markdown_table -xvs 2>&1 | tail -20
+> ```
+
+## Loss decomposition into Reasoning vs. Tool Calls
+
+We only train on assistant responses. Of the 177 million tokens in the dataset, only 46 million carry loss — the remaining 131 million (system prompts and tool responses) are masked.
+
+Within an assistant message there are two very different kinds of tokens, mainly reasoning and tool calls. Tool-call tokens are far lower-entropy:
+
+![A packed training row rendered with per-token probabilities: reasoning tokens in grey, tool call tokens in pink, masked tokens untrained, dimmer shades meaning higher probability](media/reasoning-tool-calls-logprobs.png)
+
+Although tool-call tokens make up 43.3% of input tokens, they contribute only 16.1% of the overall loss. To track convergence separately, we decompose the loss into two components: negative log-likelihood on reasoning tokens and on tool-call tokens.
+
+![Train and eval loss curves decomposed into reasoning and tool-call components with epoch boundaries; every eval loss reaches its global minimum at step 300](media/train_loss_decomposition.png)
+
+The training-loss curves show 10–20% cliffs at epoch boundaries — the usual signature of memorizing repeated data. We read it as benign: eval losses keep improving through the second epoch.
+
+It’s possible to set different learning rates for the two components, but on our data both eval losses reach their global minima at the same moment (the end of epoch 2) so we didn’t need to.
+
+## Test scores
+
+Below is the comparison of GLM-4.7-Flash, GLM-4.7-Flash-Coder, and its teacher GLM-5.1 on test set of SWE-rebench-V2:
+
+| metric | GLM-4.7-Flash | GLM-4.7-Flash-Coder | Δ | Teacher GLM-5.1 |
+| --- | --- | --- | --- | --- |
+| pass@1 | 0.3315 | 0.4165 | +25.6% | 0.5650 |
+| pass@2 | 0.4087 | 0.5023 | +22.9% | 0.6063 |
+| pass@4 | 0.4620 | 0.5720 | +23.8% | 0.6300 |
+
+## How we enabled FA4
+
+FlashAttention-4 is a fast attention kernel — but it’s incompatible with GLM-4.7-Flash’s architecture:
+
+> *FlashAttention-4 produces NaN gradients for this model’s head_dim-256 partial-rotary attention; falling back to `attn_implementation='sdpa'`.*
+
+We found that the incompatibility only appears in the backward pass; the forward computation is correct. So we designed a hybrid scheme: FA4 handles the forward pass, while FA2 takes over for the backward. The hybrid approach gave a 1.36× increase in attention throughput.
+
+*We’ll have a separate post on FA4-fwd-FA2-bwd in [our research blog](https://whitecircle.com/#research); stay tuned! 💚*
+
+## Kernels
+
+GLM-4.7-Flash is a Mixture-of-Experts model. Tokens are distributed unevenly across experts, so the MLP blocks receive tensors of varying width. The Grouped-GEMM kernel batches matrix multiplications of varying shapes into a single kernel launch. Enabling it gave a 1.6× speedup on the FFN step.
+
+Additionally, with Liger Fused Linear Cross-Entropy, we were able to compute loss without full logit matrix materialization in VRAM, preventing OOM on long sequences.
+
+Together, these improvements yield up to a 1.63× training speedup over TRL, raising throughput to 11.1k tokens/s on a single B300 GPU on context length 4k:
+
+![GLM-4.7-Flash SFT throughput on a single B300 across context lengths from 4k to 128k: Halo FA4-hybrid leads Halo FA2 and TRL at every length; TRL OOMs at 128k](media/glm47flash_throughput.png)
+
+## Agentic infra challenge
+
+Agentic evaluation at scale turned out to be a real infrastructure problem.
+
+With thousands of concurrent agents, Docker brings high startup overhead and heavy disk pressure, and quickly saturates the local network — producing a high rate of environment startup errors.
+
+Kubernetes struggled at high concurrency on a single host as well: it couldn’t set up and tear down pods fast enough, triggering timeouts. The problem was worst on powerful hosts: they could support enormous concurrency, but pod scheduling couldn’t keep up, so we never reached full CPU or RAM utilization.
+
+## Overlay sandboxes
+
+Our solution is the Overlay Sandbox: instead of a container image, the agent runs on the bare host, inside an isolated filesystem and process group.
+
+It has three layers: a base filesystem with the OS and repository, a hidden layer with the verifier suite, and an upper layer holding all edits and pip-installed packages.
+
+Because overlay mounts are lazy (they don’t even index the filesystem!) and track only modified files, sandbox startup is essentially free.
+
+![Overlay sandbox diagram: a read-only base filesystem, hidden verifier files not visible to the agent, and a writable lazy overlay mount holding edits and new files](media/overlay_sandbox.png)
+
+Since SWE-rebench-V2 is built from public repositories, agents must run without network access — otherwise, a model can simply fetch a real fix from GitHub. Before restricting network, we caught Fable 5 and Sonnet 5 doing exactly this — retrieving the upstream patches directly on 20-40% of tasks.
+
+## Traffic recorder
+
+Our sandbox also includes a traffic recorder — a tiny proxy that forwards requests to the provider (OpenRouter / vLLM / SGLang) and records every response. It’s a harness-agnostic way to assemble full trajectories in OpenAI format and collect performance metrics.
+
+![Traffic recorder diagram: a proxy between the agent and the provider records raw requests and assembles them into an OpenAI-format trace](media/traffic_recorder.png)
+
+## Evaluation
+
+Every agent runs with `max_turns = 200`, medium reasoning effort, and default sampling parameters.
+
+Inference costs are calculated from OpenRouter pricing tables. We report the theoretical price under the assumption of perfect caching — actual OpenRouter costs were higher due to inconsistent routing. For locally served GLM-4.7-Flash-Coder, we applied the per-request pricing of GLM-4.7-Flash via OpenRouter’s DeepInfra provider.
+
+<details>
+<summary>OpenRouter providers per model</summary>
+
+| model | provider | input $/Mtok | output $/Mtok | cache_read $/Mtok | cache_write $/Mtok |
+| --- | --- | --- | --- | --- | --- |
+| Fable-5 | google-vertex/global | 10 | 50 | 1 | 12.5 |
+| Opus-4.8 | anthropic | 5 | 25 | 0.5 | 6.25 |
+| GLM-5.1 | wafer/fp4 | 1 | 3.2 | 0.1 |  |
+| MiniMax-M3 | minimax/fp8 | 0.3 | 1.2 | 0.06 |  |
+| DeepSeek-V4-Pro | deepinfra/fp4 | 1.3 | 2.6 | 0.1 |  |
+| GPT-5.5 | azure | 5 | 30 | 0.5 |  |
+| Kimi-K2.6 | siliconflow/fp8 | 0.77 | 3.4 | 0.14 |  |
+| Haiku-4.5 | anthropic | 1 | 5 | 0.1 | 1.25 |
+| Qwen3.5-397B-A17B | deepinfra/fp8 | 0.45 | 3 | 0.22 |  |
+| GPT-5.4-Mini | azure | 0.75 | 4.5 | 0.075 |  |
+| Gemini-3.1-Pro | google-ai-studio | 2 | 12 | 0.2 | 0.375 |
+| GLM-4.7-Flash-Coder | deepinfra/bf16 | 0.06 | 0.4 | 0.01 |  |
+| Ling-2.6-Flash | novita | 0.1 | 0.3 | 0.02 |  |
+| Devstral-2 | mistral | 0.4 | 2 | 0.04 |  |
+| Qwen3.5-122B-A10B | alibaba | 0.4 | 3.2 |  |  |
+| GLM-4.7-Flash | deepinfra/bf16 | 0.06 | 0.4 | 0.01 |  |
+| Gemini-3.5-Flash | google-vertex/global | 1.5 | 9 | 0.15 | 0.08333 |
+| Qwen3-32B | deepinfra/fp8 | 0.08 | 0.28 |  |  |
+| Qwen3.6-35B-A3B | parasail/fp8 | 0.15 | 1 | 0.05 |  |
+| Gemini-2.5-Flash | google-vertex/global | 0.3 | 2.5 | 0.03 | 0.08333 |
+| Nemotron-3-Super | digitalocean | 0.21 | 0.455 | 0.06 |  |
+| Solar-Pro-3 | upstage | 0.15 | 0.6 | 0.015 |  |
+
+</details>
+
+<details>
+<summary>Prompt template</summary>
+
+```markdown
+You are solving a real GitHub issue in the `{repo}` repository. The repo is already cloned and set up in the current working directory.
+
+## Workflow
+
+1. **Understand the issue.** Read the problem statement carefully. Identify the expected vs actual behavior.
+2. **Explore the codebase.** Find the relevant files using grep, find, or by reading the project structure. Read the code that needs to change.
+3. **Reproduce the bug** if possible — run the relevant test file or write a small script to confirm the issue exists.
+4. **Implement a minimal fix.** Change only what is necessary. Do not refactor unrelated code, do not add features beyond what the issue asks for.
+5. **Verify your fix.** Run the project's existing test suite to make sure you haven't introduced regressions. Look at the project's test configuration (Makefile, tox.ini, pyproject.toml, CI config) to find the right test command.
+6. **Mark the task complete** only after you've verified existing tests pass with your changes.
+
+## Issue
+
+{problem_statement}
+
+## Expected Interfaces
+
+The following interfaces are expected by the test suite. Pay close attention to function names, parameter signatures, and return types — the tests verify these exactly.
+
+{interface}
+```
+
+</details>
+
+<details>
+<summary>Inference with SGLang 0.5.14, CUDA 13</summary>
+
+```bash
+python3 -m sglang_router.launch_server \
+  --model-path whitecircle/GLM-4.7-Flash-Coder \
+  --trust-remote-code \
+  --host 0.0.0.0 --port 30000 \
+  --context-length 202752 \
+  --prefill-attention-backend trtllm_mla \
+  --decode-attention-backend triton \
+  --page-size 64 \
+  --tool-call-parser glm47 --reasoning-parser glm45 \
+  --router-policy cache_aware \
+  --tp-size 1 --dp-size 8
+```
+
+</details>
+
+## Team
+
+- Project lead: Oleg Smirnov
+- Agentic sandboxes: Konstantin Korolev, Sergei Bratchnikov
+- Contributors: Mohamed Mekkouri, Dmitrii Kharlapenko, Daniil Oblakov
+
+A [White Circle](https://whitecircle.com) project

chat_template.jinja

@@ -0,0 +1,86 @@
+[gMASK]<sop>
+{%- if tools -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{{ tool | tojson(ensure_ascii=False) }}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) %}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {% set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{% for m in messages %}
+{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set reasoning_content = '' %}
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- else %}
+    {%- if '</think>' in content %}
+        {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+        {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+    {%- endif %}
+{%- endif %}
+{%- if ((clear_thinking is defined and not clear_thinking) or loop.index0 > ns.last_user_index) and reasoning_content -%}
+{{ '<think>' + reasoning_content.strip() +  '</think>'}}
+{%- else -%}
+{{ '</think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{- '<tool_call>' + tc.name -}}
+{% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if m.content is string -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' }}
+{%- endif %}
+{{- '<tool_response>' }}
+{{- m.content }}
+{{- '</tool_response>' }}
+{%- else -%}
+<|observation|>{% for tr in m.content %}
+<tool_response>{{ tr.output if tr.output is defined else tr }}</tool_response>{% endfor -%}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '</think>' if (enable_thinking is defined and not enable_thinking) else '<think>' -}}
+{%- endif -%}

config.json

@@ -0,0 +1,319 @@
+{
+  "architectures": [
+    "Glm4MoeLiteForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "pad_token_id": 154820,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "intermediate_size": 10240,
+  "max_position_embeddings": 202752,
+  "model_type": "glm4_moe_lite",
+  "moe_intermediate_size": 1536,
+  "topk_method": "noaux_tc",
+  "norm_topk_prob": true,
+  "num_attention_heads": 20,
+  "n_group": 1,
+  "topk_group": 1,
+  "n_routed_experts": 64,
+  "n_shared_experts": 1,
+  "routed_scaling_factor": 1.8,
+  "num_experts_per_tok": 4,
+  "first_k_dense_replace": 1,
+  "num_hidden_layers": 47,
+  "num_key_value_heads": 20,
+  "num_nextn_predict_layers": 1,
+  "partial_rotary_factor": 1.0,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "tie_word_embeddings": false,
+  "dtype": "bfloat16",
+  "transformers_version": "5.0.0rc0",
+  "q_lora_rank": 768,
+  "kv_lora_rank": 512,
+  "qk_nope_head_dim": 192,
+  "qk_rope_head_dim": 64,
+  "v_head_dim": 256,
+  "vocab_size": 154880,
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "input_activations": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "weights": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "targets": [
+          "model.layers.1.mlp.experts",
+          "model.layers.2.mlp.experts",
+          "model.layers.3.mlp.experts",
+          "model.layers.4.mlp.experts",
+          "model.layers.5.mlp.experts",
+          "model.layers.6.mlp.experts",
+          "model.layers.7.mlp.experts",
+          "model.layers.8.mlp.experts",
+          "model.layers.9.mlp.experts",
+          "model.layers.10.mlp.experts",
+          "model.layers.11.mlp.experts",
+          "model.layers.12.mlp.experts",
+          "model.layers.13.mlp.experts",
+          "model.layers.14.mlp.experts",
+          "model.layers.15.mlp.experts",
+          "model.layers.16.mlp.experts",
+          "model.layers.17.mlp.experts",
+          "model.layers.18.mlp.experts",
+          "model.layers.19.mlp.experts",
+          "model.layers.20.mlp.experts",
+          "model.layers.21.mlp.experts",
+          "model.layers.22.mlp.experts",
+          "model.layers.23.mlp.experts",
+          "model.layers.24.mlp.experts",
+          "model.layers.25.mlp.experts",
+          "model.layers.26.mlp.experts",
+          "model.layers.27.mlp.experts",
+          "model.layers.28.mlp.experts",
+          "model.layers.29.mlp.experts",
+          "model.layers.30.mlp.experts",
+          "model.layers.31.mlp.experts",
+          "model.layers.32.mlp.experts",
+          "model.layers.33.mlp.experts",
+          "model.layers.34.mlp.experts",
+          "model.layers.35.mlp.experts",
+          "model.layers.36.mlp.experts",
+          "model.layers.37.mlp.experts",
+          "model.layers.38.mlp.experts",
+          "model.layers.39.mlp.experts",
+          "model.layers.40.mlp.experts",
+          "model.layers.41.mlp.experts",
+          "model.layers.42.mlp.experts",
+          "model.layers.43.mlp.experts",
+          "model.layers.44.mlp.experts",
+          "model.layers.45.mlp.experts",
+          "model.layers.46.mlp.experts"
+        ]
+      }
+    },
+    "ignore": [
+      "lm_head",
+      "model.embed_tokens",
+      "model.layers.*.mlp.down_proj",
+      "model.layers.*.mlp.gate",
+      "model.layers.*.mlp.gate_proj",
+      "model.layers.*.mlp.shared_experts.down_proj",
+      "model.layers.*.mlp.shared_experts.gate_proj",
+      "model.layers.*.mlp.shared_experts.up_proj",
+      "model.layers.*.mlp.up_proj",
+      "model.layers.*.self_attn.kv_a_proj_with_mqa",
+      "model.layers.*.self_attn.kv_b_proj",
+      "model.layers.*.self_attn.o_proj",
+      "model.layers.*.self_attn.q_a_proj",
+      "model.layers.*.self_attn.q_b_proj"
+    ],
+    "quant_algo": "MIXED_PRECISION",
+    "producer": {
+      "name": "modelopt",
+      "version": "0.47.0rc1.dev129+g333ace1bc"
+    },
+    "quant_method": "modelopt",
+    "quantized_layers": {
+      "model.layers.1.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.10.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.11.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.12.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.13.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.14.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.15.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.16.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.17.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.18.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.19.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.2.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.20.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.21.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.22.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.23.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.24.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.25.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.26.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.27.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.28.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.29.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.3.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.30.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.31.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.32.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.33.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.34.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.35.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.36.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.37.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.38.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.39.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.4.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.40.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.41.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.42.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.43.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.44.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.45.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.46.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.5.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.6.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.7.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.8.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.9.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      }
+    }
+  }
+}

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "pad_token_id": 154820,
+  "temperature": 1.0,
+  "transformers_version": "5.0.0.dev0"
+}

hf_quant_config.json

@@ -0,0 +1,213 @@
+{
+  "producer": {
+    "name": "modelopt",
+    "version": "0.47.0rc1.dev129+g333ace1bc"
+  },
+  "quantization": {
+    "quant_algo": "MIXED_PRECISION",
+    "kv_cache_quant_algo": null,
+    "group_size": 16,
+    "exclude_modules": [
+      "lm_head",
+      "model.embed_tokens",
+      "model.layers.*.mlp.down_proj",
+      "model.layers.*.mlp.gate",
+      "model.layers.*.mlp.gate_proj",
+      "model.layers.*.mlp.shared_experts.down_proj",
+      "model.layers.*.mlp.shared_experts.gate_proj",
+      "model.layers.*.mlp.shared_experts.up_proj",
+      "model.layers.*.mlp.up_proj",
+      "model.layers.*.self_attn.kv_a_proj_with_mqa",
+      "model.layers.*.self_attn.kv_b_proj",
+      "model.layers.*.self_attn.o_proj",
+      "model.layers.*.self_attn.q_a_proj",
+      "model.layers.*.self_attn.q_b_proj"
+    ],
+    "quantized_layers": {
+      "model.layers.1.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.10.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.11.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.12.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.13.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.14.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.15.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.16.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.17.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.18.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.19.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.2.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.20.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.21.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.22.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.23.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.24.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.25.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.26.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.27.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.28.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.29.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.3.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.30.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.31.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.32.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.33.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.34.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.35.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.36.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.37.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.38.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.39.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.4.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.40.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.41.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.42.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.43.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.44.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.45.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.46.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.5.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.6.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.7.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.8.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      },
+      "model.layers.9.mlp.experts": {
+        "quant_algo": "NVFP4",
+        "group_size": 16
+      }
+    }
+  }
+}

model-00001-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e248d1280c7c43b438f2c83336ae591744062baee4221dbbdba0a2f426c26b6a
+size 1258432136

model-00002-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e58e86980313e3930dc5c4b91bbc9258dfeaa9f870bcd5b1c687afdd3524fbc
+size 1258432136

model-00003-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42a7ae24de58c0d237e589d10219bcafe0f50fb203ef5cc6407c71d521b59ed4
+size 1258433792

model-00004-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e95e14b442809d0cac9ce66b1af7e608fca69973fce8081ac75ecbe400335ca9
+size 1258435064

model-00005-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e8a5e780fbd7618e8a38ab96eba9dcaf4236317eeadc02d7aa5136b064cb9df
+size 1258434976

model-00006-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a29e568000249bacc57915ddf4c23549317684897ab0098ed6a1d1b74d92b1fe
+size 1258434976

model-00007-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:236e95bab994248adcb2371477078a0bd8176fd9f937ba8e48f0a787c16fe9ec
+size 1258435096

model-00008-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee525e3a2fba3b9d551774951dd2b9ce53e128daef873bece11a7e449ddff284
+size 1258434976

model-00009-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d09397b6935e92a9ddae46a00b6e389ff16e77b5f91ef7b5178202ac6e822a4
+size 1258434976

model-00010-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39accd9bafc25c53f95c38ecc440ac890a73362f99dd85e928b7d762038b3b94
+size 1258435072

model-00011-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b602a470b4e8b4ffbbfd18039f13474bc385866734b63df7c035c2ac0fd34b12
+size 1258435000

model-00012-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1687587c15ab407cb16ffcff05d00a2226ac23c9d099644f43fa5493fe25eeb3
+size 1258434976

model-00013-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40761f9e7cf5ecf8c3e4cdfd9a5f267cd514b49060a4161af267e40198321056
+size 1165373816

model-00014-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:559b7d4f895176c0c978f52b14bc9590a6a7d70f69724a08b7fb7b304901adc2
+size 1258169944

model-00015-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e25b882b0b98297582aaad90d7fff376dc93b9cfaa48417f7763f4c0c5ea56c1
+size 1254429808

model-00016-of-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7247f97d8c6eac2b25c1cc6737ad3ea4150e8f99bf43ce6a9d76b0955acea9f
+size 1173744304

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d
+size 20217442

tokenizer_config.json

@@ -0,0 +1,321 @@
+{
+  "added_tokens_decoder": {
+    "154820": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154821": {
+      "content": "[MASK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154822": {
+      "content": "[gMASK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154823": {
+      "content": "[sMASK]",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154824": {
+      "content": "<sop>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154825": {
+      "content": "<eop>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154826": {
+      "content": "<|system|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154827": {
+      "content": "<|user|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154828": {
+      "content": "<|assistant|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154829": {
+      "content": "<|observation|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154830": {
+      "content": "<|begin_of_image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154831": {
+      "content": "<|end_of_image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154832": {
+      "content": "<|begin_of_video|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154833": {
+      "content": "<|end_of_video|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154834": {
+      "content": "<|begin_of_audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154835": {
+      "content": "<|end_of_audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154836": {
+      "content": "<|begin_of_transcription|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154837": {
+      "content": "<|end_of_transcription|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "154838": {
+      "content": "<|code_prefix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154839": {
+      "content": "<|code_middle|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154840": {
+      "content": "<|code_suffix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154841": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154842": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154843": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154844": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154845": {
+      "content": "<tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154846": {
+      "content": "</tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154847": {
+      "content": "<arg_key>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154848": {
+      "content": "</arg_key>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154849": {
+      "content": "<arg_value>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154850": {
+      "content": "</arg_value>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154851": {
+      "content": "/nothink",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154852": {
+      "content": "<|begin_of_box|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154853": {
+      "content": "<|end_of_box|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154854": {
+      "content": "<|image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "154855": {
+      "content": "<|video|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|begin_of_audio|>",
+    "<|end_of_audio|>",
+    "<|begin_of_transcription|>",
+    "<|end_of_transcription|>"
+  ],
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 128000,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "remove_space": false,
+  "tokenizer_class": "PreTrainedTokenizer"
+}

verification_report.txt

@@ -0,0 +1,26 @@
+verification report (v5.2)
+[PASS] INDEX: 35988 tensors, index consistent
+[PASS] CONFIG: quant_algo MIXED_PRECISION
+[PASS] QUANT_TENSORS: 8832 quantized modules complete (['nvfp4'])
+[PASS] DECLARED: 46 quantized units declared with matching algorithms
+[PASS] EXCLUDE: 14 patterns, none matches a quantized module
+[PASS] BASE: all 9492 base tensors present, converted or replaced by quantized modules
+[PASS] DEEP_SCALES: 17664 F32 scale values finite and positive
+[PASS] FUSED_SCALES: 2944 q/k/v and gate/up groups share one global weight scale
+[PASS] EXPERTS: all 46 MoE layers x 64 experts x 3 projections (down_proj/gate_proj/up_proj) complete, NVFP4 (U8 packed weights, F8 block scales, F32 global + input scales)
+0 FAIL | 0 WARN | 9 PASS
+--- checkpoint inventory ---
+[INFO] FILES: 16 model shards + 0 PLE shards | 35,988 tensors | 18.58 GiB
+[INFO] DENSE LAYERS: [0] carry no routed experts (MLP BF16)
+[INFO] ROUTERS: 46 router weights BF16 [64, 2048] + 47 correction biases (never quantized)
+[INFO] SHARED EXPERTS: 138 tensors BF16 across 46 layers
+[INFO] ATTENTION: 47 layers, 329 projections BF16 (all unquantized)
+[INFO] MTP: 1 tensors, 0.00 GiB (no routed experts)
+[INFO] EMBED_TOKENS: [154880, 2048] BF16, 0.59 GiB
+[INFO] LM_HEAD: [154880, 2048] BF16, 0.59 GiB
+[INFO] SIZE: routed experts 14.55 GiB | attention 1.90 GiB | embeddings/head 1.18 GiB | shared experts 0.81 GiB | other (norms, dense mlp, ...) 0.12 GiB | routers 0.01 GiB | mtp 0.00 GiB
+[INFO] DECLARED: quantized_layers NVFP4 x46
+[INFO] KV CACHE: not declared (serve with --kv-cache-dtype fp8 if wanted)
+[INFO] EXCLUSIONS (14): lm_head, model.embed_tokens, model.layers.*.mlp.down_proj, model.layers.*.mlp.gate, model.layers.*.mlp.gate_proj, model.layers.*.mlp.shared_experts.down_proj, model.layers.*.mlp.shared_experts.gate_proj, model.layers.*.mlp.shared_experts.up_proj, model.layers.*.mlp.up_proj, model.layers.*.self_attn.kv_a_proj_with_mqa, model.layers.*.self_attn.kv_b_proj, model.layers.*.self_attn.o_proj, model.layers.*.self_attn.q_a_proj, model.layers.*.self_attn.q_b_proj
+[INFO] SCALE RANGES: weight_scale_2 (8832): min 3.924e-05 / median 5.050e-05 / max 2.296e-04
+[INFO] SCALE RANGES: input_scale (8832): min 6.285e-05 / median 3.698e-04 / max 8.324e-03