Duplicate from nvidia/GLM-5.3-Flash-FP4-Local

Co-authored-by: Luke Alonso <nv-lukea@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,3 @@
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,490 @@
+---
+pipeline_tag: image-text-to-text
+base_model:
+- zai-org/GLM-5.3-Flash
+license: other
+license_name: nvidia-open-model-agreement
+library_name: Model Optimizer
+tags:
+- nvidia
+- ModelOpt
+- GLM-5
+- quantized
+- FP4
+- fp4
+---
+
+# Model Overview
+
+**IMPORTANT**: the NVFP4 scale factors in this checkpoint are in a new lossless compression format, but upstream vLLM support for this format has not yet landed. **You must use one of the containers provided below.**
+
+## Description:
+The NVIDIA GLM-5.3-Flash-FP4-Local model is the quantized version of ZAI's GLM-5.3-Flash model, which is an auto-regressive language model that uses an optimized transformer architecture. GLM-5.3-Flash is a natively multimodal Mixture-of-Experts (MoE) model for reasoning, coding, and agentic tasks; it uses a hybrid sparse and linear attention architecture with Manifold-Constrained Hyper-Connections (mHC) to support a long context. For more information, please check [here](https://huggingface.co/zai-org/GLM-5.3-Flash). The NVIDIA GLM-5.3-Flash-FP4-Local model is quantized with [Model Optimizer](https://github.com/NVIDIA/Model-Optimizer) using [quantization-aware distillation (QAD)](https://developer.nvidia.com/blog/how-quantization-aware-training-enables-low-precision-accuracy-recovery/).
+
+This checkpoint is best suited for local AI hardware and uses approximately **166 GiB of vRAM**. It has been validated with vLLM on the following hardware: <br>
+* **2 x NVIDIA RTX PRO 6000 Blackwell with 1.8M tokens of FP8 KV Cache** <br>
+* **2 x NVIDIA DGX Spark 128 GB with 2M tokens of FP8 KV Cache** <br>
+
+This model is ready for commercial or non-commercial use.  <br>
+
+## Third-Party Community Consideration
+This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party's requirements for this application and use case; see link to Non-NVIDIA [(GLM-5.3-Flash) Model Card](https://huggingface.co/zai-org/GLM-5.3-Flash) from ZAI.
+
+### License/Terms of Use:
+**GOVERNING DOWNLOAD TERMS:** Use of this model is governed by [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/).
+
+**ADDITIONAL INFORMATION** : The GLM-5.3-Flash base model is governed by the [MIT License](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) . **GLM-5.3-Flash**).
+
+### Deployment Geography:
+Global <br>
+
+### Use Case: <br>
+Developers looking to take off-the-shelf, pre-quantized models for deployment in AI Agent systems, chatbots, RAG systems, and other AI-powered applications. <br>
+
+### Release Date:  <br>
+Huggingface 10/05/2026 via https://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local <br>
+
+## References
+NVIDIA Model Optimizer: https://github.com/NVIDIA/Model-Optimizer
+
+## Model Architecture:
+**Architecture Type:** Transformers  <br>
+**Network Architecture:** Hybrid sparse and linear attention with Manifold-Constrained Hyper-Connections (`Glm5NextForConditionalGeneration`) <br>
+**Number of Model Parameters:** 320B in total and 18B activated <br>
+**This model was developed based on [GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash)** <br>
+
+## Input:
+**Input Type(s):** Text, Image, Video <br>
+**Input Format(s):** String, Red, Green, Blue (RGB), Video (MP4/WebM) <br>
+**Input Parameters:** One-Dimensional (1D), Two-Dimensional (2D), Three-Dimensional (3D) <br>
+**Other Properties Related to Input:** Context length up to 1M <br>
+
+## Output:
+**Output Type(s):** Text <br>
+**Output Format:** String <br>
+**Output Parameters:** One-Dimensional (1D): Sequences <br>
+**Other Properties Related to Output:** None <br>
+
+Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
+
+## Software Integration:
+**Supported Runtime Engine(s):** <br>
+* **vLLM** <br>
+
+**Supported Hardware Microarchitecture Compatibility:** <br>
+* NVIDIA Blackwell <br>
+
+**Preferred Operating System(s):** <br>
+* Linux <br>
+
+The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
+
+## Model Version(s):
+The model version is FP4 QAD 1.0 and is quantized with nvidia-modelopt **v0.47.0** <br>
+
+## Training and Evaluation Datasets:
+
+## Training Dataset:
+**Link**: Subset of [Nemotron-Post-Training-Dataset-v3](https://huggingface.co/collections/nvidia/nemotron-post-training-v3), [Nemotron-Post-Training-Dataset-v2](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v2) and subset of[Nemotron-Pretraining-Dataset](https://huggingface.co/collections/nvidia/nemotron-pre-training-datasets) <br> only prompts from the datasets were used for data synthesis.
+**Data Modality:** Text <br>
+**Text Training Data Size:** 500 Million Tokens, repeated for 3 epochs <br>
+**Data Collection Method by dataset:** Hybrid: Automated, Manually-Collected, Synthetic <br>
+**Labeling Method by dataset:** Hybrid: Automated, Manually-Labeled, Synthetic <br>
+**Properties:** The GLM-5.3-Flash-FP4-Local model was trained exclusively on a post-training corpus includes synthetic data for reasoning, code, science, tool use, instruction following, structured outputs, and multilingual tasks. Dataset disclosures are published in the `nvidia/nemotron-post-training-v2` and subset of `nvidia/nemotron-post-training-v3` and subset of `nvidia/nemotron-pre-training-datasets` Hugging Face collections. <br>
+
+## Evaluation
+
+**Datasets:** GPQA Diamond, ToolEvalBench <br>
+**Data Collection Method by dataset:** Hybrid: Automated, Manually-Collected <br>
+**Labeling Method by dataset:** Hybrid: Manually-Labeled, Automated <br>
+**Properties:** GPQA Diamond contains 198 graduate-level multiple-choice questions written by domain experts in biology, physics, and chemistry. ToolEvalBench is a tool-calling quality benchmark that runs deterministic multi-turn agent scenarios against mock tools, scoring tool selection, parameter precision, multi-step chains, error recovery, and structured output. <br>
+
+The accuracy benchmark results are presented in the table below:
+
+| Precision | GPQA Diamond | ToolEvalBench |
+|:---|:---:|:---:|
+| BF16 | 92.1% | 91.9% |
+| **FP4-Local** | **91.3%** | **92.4%** |
+
+> Baseline: [GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash).
+
+## Quantization-Aware Distillation
+The BF16 teacher generates chat responses and supplies probability and hidden-state targets for the student.
+
+During training, the student's routed-expert weights are quantized to NVFP4 and reconstructed for each forward pass. Gradients update the underlying BF16 weights so their low-precision representations better match the teacher's next-token probabilities and final hidden states.
+
+The student uses its own expert routing during training. Routers, shared experts, dense MLPs and normalization weights are also adapted, allowing the unquantized components to compensate alongside the NVFP4 experts.
+
+### What's quantized
+
+| Component | Stored precision | Distillation |
+|:---|:---|:---|
+| Decoder routed experts | QAD NVFP4; E2M1 values, E4M3 block scales, FP32 global/input scales | Trained |
+| Decoder attention | MXFP8 projection selection from the pinned source | Trained |
+| MTP routed experts | NVFP4, BF16 activations; block scales stored directly | Trained |
+| Vision attention linears | MXFP8 E4M3 values; UE8M0 scales per 32 weights; 48 matrices | Trained |
+| Vision MLP and merger linears | NVFP4 E2M1 values; E4M3 scales per 16 weights and FP32 global scale; 76 matrices; BF16 activations | Updated during distillation |
+| Other vision tensors | BF16; 223 tensors, including normalization, biases, position embeddings, and convolutions | Frozen |
+| Normalization parameters | 277 BF16 tensors; original bytes preserved | Unchanged; outside text distillation |
+
+## Inference:
+**Acceleration Engine:** **vLLM** <br>
+**Test Hardware:** **NVIDIA DGX Spark and NVIDIA RTX PRO 6000 Blackwell** <br>
+
+## Usage
+
+This checkpoint requires community vLLM containers (provided in cooperation with the "Local Inference Lab" community organization) until all features are available in upstream vLLM.
+
+### 2 x NVIDIA DGX Spark
+
+vLLM container: [`eugr/spark-vllm-b12x:latest`](https://hub.docker.com/r/eugr/spark-vllm-b12x/tags?name=latest)
+
+One step launcher:
+
+```sh
+git clone https://github.com/eugr/spark-vllm-docker
+cd spark-vllm-docker
+./run-recipe.sh recipes/glm-5.3-flash.yaml --setup
+```
+
+Manual launch:
+
+Start the container on both nodes. Set `HF_TOKEN` in the host environment, and set the network values below to match your setup.
+
+```sh
+# Example two-Spark values (adjust to your network):
+#   HEAD_IP="10.100.32.1"        # head node IP on the ETH subnet
+#   ETH_IF="enp1s0f0np0"        # ConnectX baseboard ethernet (socket/control)
+#   IB_IF="rocep1s0f0,roceP2p1s0f0"   # RoCE HCA names (data plane) - use both "twins"!
+export HEAD_IP="<head-connectx-ip>"
+export ETH_IF="<connectx-interface>"
+export IB_IF="<roce-interface-1>,<roce-interface-2>"
+export MASTER_PORT=29501
+export IMAGE="eugr/spark-vllm-b12x:latest"
+export HF_CACHE_DIR="$HOME/.cache/huggingface"
+
+mkdir -p "$HF_CACHE_DIR" "$HOME/.cache/vllm" "$HOME/.cache/flashinfer" "$HOME/.triton" "$HOME/.cache/b12x"
+
+docker run --rm -it \
+  --privileged \
+  --gpus all \
+  --ipc host \
+  --network host \
+  --shm-size=16g \
+  --ulimit nofile=1048576:1048576 \
+  --ulimit memlock=-1 \
+  --ulimit stack=67108864 \
+  -e HF_TOKEN=$HF_TOKEN \
+  -e MN_IF_NAME="$ETH_IF" \
+  -e UCX_NET_DEVICES="$ETH_IF" \
+  -e NCCL_SOCKET_IFNAME="$ETH_IF" \
+  -e NCCL_IB_HCA="$IB_IF" \
+  -e NCCL_IB_DISABLE=0 \
+  -e OMPI_MCA_btl_tcp_if_include="$ETH_IF" \
+  -e GLOO_SOCKET_IFNAME="$ETH_IF" \
+  -e TP_SOCKET_IFNAME="$ETH_IF" \
+  -e NCCL_IGNORE_CPU_AFFINITY=1 \
+  -e HEAD_IP="$HEAD_IP" \
+  -e MASTER_PORT="$MASTER_PORT" \
+  -v "$HF_CACHE_DIR:/root/.cache/huggingface" \
+  -v "$HOME/.cache/vllm:/root/.cache/vllm" \
+  -v "$HOME/.cache/flashinfer:/root/.cache/flashinfer" \
+  -v "$HOME/.triton:/root/.triton" \
+  -v $HOME/.cache/b12x:/root/.cache/b12x \
+  --entrypoint /bin/bash \
+  "$IMAGE"
+```
+
+Both nodes share the same environment:
+
+```sh
+export CUTE_DSL_ARCH="sm_121a"
+export SAFETENSORS_FAST_GPU="1"
+export VLLM_ENABLE_ROCE_ALLREDUCE="1"
+export VLLM_ROCE_ALLREDUCE_MAX_SIZE="2MB"
+export NCCL_BUFFSIZE=1048576
+export NCCL_MAX_CTAS=8
+export VLLM_WORKER_MULTIPROC_METHOD="spawn"
+export VLLM_SSM_CONV_STATE_LAYOUT="DS"
+export VLLM_USE_AOT_COMPILE="1"
+export VLLM_USE_MEGA_AOT_ARTIFACT="1"
+export VLLM_USE_V2_MODEL_RUNNER="1"
+export VLLM_ENABLE_PCIE_ALLREDUCE="0"
+export B12X_POLICY_MODE="auto"
+export VLLM_B12X_MOE_FP4_FORCE_A16=1
+export B12X_W4A16_A4_PREFILL=1
+```
+
+Launch the head node:
+
+```sh
+vllm serve nvidia/GLM-5.3-Flash-FP4-Local \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --tensor-parallel-size 2 \
+  --mamba-cache-mode align \
+  --enable-prefix-caching \
+  --enable-chunked-prefill \
+  --dtype bfloat16 \
+  --kv-cache-dtype fp8 \
+  --quantization modelopt_mixed \
+  --attention-backend B12X \
+  --block-size 256 \
+  --moe-backend b12x \
+  --linear-backend b12x \
+  --no-enable-flashinfer-autotune \
+  --load-format b12x \
+  --max-model-len 500000 \
+  --max-num-seqs 4 \
+  --max-num-batched-tokens 4096 \
+  --speculative-config '{"method":"mtp","num_speculative_tokens":5,"moe_backend":"humming","attention_backend":"B12X"}' \
+  --reasoning-parser glm45 \
+  --tool-call-parser glm47 \
+  --enable-auto-tool-choice \
+  --gpu-memory-utilization 0.87 \
+  --nnodes 2 \
+  --node-rank 0 \
+  --master-addr "$HEAD_IP" \
+  --master-port "$MASTER_PORT"
+```
+
+Launch the worker node:
+
+```sh
+vllm serve nvidia/GLM-5.3-Flash-FP4-Local \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --tensor-parallel-size 2 \
+  --mamba-cache-mode align \
+  --enable-prefix-caching \
+  --enable-chunked-prefill \
+  --dtype bfloat16 \
+  --kv-cache-dtype fp8 \
+  --quantization modelopt_mixed \
+  --attention-backend B12X \
+  --block-size 256 \
+  --moe-backend b12x \
+  --linear-backend b12x \
+  --no-enable-flashinfer-autotune \
+  --load-format b12x \
+  --max-model-len 500000 \
+  --max-num-seqs 4 \
+  --max-num-batched-tokens 4096 \
+  --speculative-config '{"method":"mtp","num_speculative_tokens":5,"moe_backend":"humming","attention_backend":"B12X"}' \
+  --reasoning-parser glm45 \
+  --tool-call-parser glm47 \
+  --enable-auto-tool-choice \
+  --gpu-memory-utilization 0.87 \
+  --nnodes 2 \
+  --node-rank 1 \
+  --master-addr "$HEAD_IP" \
+  --master-port "$MASTER_PORT" \
+  --headless
+```
+
+> Use the same CX7 port on every node; assign both port subinterfaces on separate subnets; cache the image and checkpoint on every node; start workers before the head. Docker runs privileged.
+
+### 2 x NVIDIA RTX PRO 6000 Blackwell
+
+vLLM container: [`ghcr.io/local-inference-lab/vllm:release-10142026`](https://github.com/local-inference-lab/blackwell-llm-docker/pkgs/container/vllm)
+
+Simplified run command:
+
+```sh
+  docker run -d --name glm53-tp2 --init --gpus '"device=0,1"' \
+    --restart on-failure --network host --ipc host --shm-size 32g --stop-timeout 30 \
+    --ulimit memlock=-1 --ulimit stack=67108864:67108864 \
+    -v model-cache:/root/.cache/huggingface -v glm53-runtime:/cache \
+    -e PRESET=glm53-tp2 -e PORT=8000 \
+    ghcr.io/local-inference-lab/vllm:release-10152026
+```
+
+Two step verbose commands:
+
+Start the container. Set `HF_TOKEN` in the host environment first.
+
+```sh
+docker run --rm -it \
+  --gpus all \
+  --ipc host \
+  --network host \
+  --shm-size=16g \
+  --ulimit memlock=-1 \
+  --ulimit stack=67108864 \
+  -e HF_TOKEN=$HF_TOKEN \
+  -v ~/.cache/huggingface:/root/.cache/huggingface \
+  --entrypoint /bin/bash \
+  ghcr.io/local-inference-lab/vllm:release-10152026
+```
+
+Then launch vLLM inside the container:
+
+```sh
+export B12X_DYNAMIC_DIRECT_EXPERT_SCALES=1
+export B12X_DYNAMIC_SKIP_SPLIT_BARRIER_RESET=1
+export B12X_DYNAMIC_SPLIT_COMPUTE_MAC=224
+export B12X_DYNAMIC_SPLIT_FAST_PREPARE=1
+export B12X_DYNAMIC_SPLIT_LOW_SMEM=1
+export B12X_DYNAMIC_SPLIT_ROUTE_COMPUTE=1
+export B12X_DYNAMIC_WORK_SOURCE=persistent_grid
+export B12X_MHC_PDL=1
+export B12X_PCIE_ONESHOT_BLOCK_LIMIT=4
+export B12X_PCIE_ONESHOT_PDL=1
+export B12X_PCIE_ONESHOT_THREADS=512
+export B12X_W4A16_A4_PREFILL=1
+export B12X_W4A16_FP32_TOPK_WEIGHTS=1
+export CUBLAS_WORKSPACE_CONFIG=:4096:1
+export CUDA_DEVICE_ORDER=PCI_BUS_ID
+export CUTE_DSL_ARCH=sm_120a
+export GLOO_SOCKET_IFNAME=lo
+export INSTANTTENSOR_BACKEND=BUFFERED
+export NCCL_BUFFSIZE=1048576
+export NCCL_IB_DISABLE=1
+export NCCL_MAX_NCHANNELS=2
+export NCCL_MIN_NCHANNELS=2
+export NCCL_NET_PLUGIN=none
+export NCCL_P2P_LEVEL=SYS
+export NCCL_PROTO=LL,LL128,Simple
+export NCCL_SOCKET_IFNAME=lo
+export NCCL_TUNER_PLUGIN=none
+export OMP_NUM_THREADS=1
+export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,large_segment_size_mb:12
+export SAFETENSORS_FAST_GPU=1
+export TORCH_CUDA_ARCH_LIST=12.0a
+export VLLM_B12X_MLA_CKV_GATHER=1
+export VLLM_B12X_MLA_CKV_GATHER_MAX_TOKENS=65536
+export VLLM_B12X_MOE_FP4_FORCE_A16=1
+export VLLM_CAUSAL_CONV1D_UPDATE_HOIST=1
+export VLLM_DISABLED_KERNELS=MarlinFP8ScaledMMLinearKernel
+export VLLM_DISABLE_SHARED_EXPERTS_STREAM=0
+export VLLM_ENABLE_PCIE_ALLREDUCE=1
+export VLLM_GLM53_DFLASH_ATTN=1
+export VLLM_GLM53_EMBED_HOST=1
+export VLLM_GLM53_KDA_GATE_SIDE_STREAM=1
+export VLLM_GLM53_KDA_MXFP8_ACTIVATION_MODE=prefill
+export VLLM_GLM53_L2_PREFETCH=1
+export VLLM_GLM53_L2_PREFETCH_PERSIST_MB=0
+export VLLM_GLM53_MTP_DRAFT_HEAD=bf16
+export VLLM_GLM53_ONLINE_DENSE_MXFP8=0
+export VLLM_GLM53_SPLIT_MAMBA_BLOCK_SIZE=auto
+export VLLM_GLM53_SPLIT_TARGET_BLOCK_SIZE=2048
+export VLLM_LM_HEAD_A16=1
+export VLLM_MTP_NVFP4_LM_HEAD=0
+export VLLM_MXFP8_LM_HEAD=0
+export VLLM_PCIE_ALLREDUCE_BACKEND=b12x
+export VLLM_PCIE_DMA_MIN_BYTES=6MB
+export VLLM_PCIE_TWOSHOT_ALLREDUCE_MAX_SIZE=off
+export VLLM_PLUGINS=''
+export VLLM_SHARE_PYNCCL_COMMS=1
+export VLLM_USE_FASTOKENS=1
+export VLLM_USE_FLASHINFER_SAMPLER=1
+export VLLM_USE_V2_MODEL_RUNNER=1
+export VLLM_WORKER_MULTIPROC_METHOD=spawn
+
+vllm serve nvidia/GLM-5.3-Flash-FP4-Local \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --additional-config '{"glm53_kda_decode_backend":"auto","kda_prefill_backend":"b12x"}' \
+  --attention-backend B12X \
+  --block-size 256 \
+  --chat-template /opt/lil/runtime/templates/glm53-flash.jinja \
+  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE"}' \
+  --cp-kv-cache-interleave-size 4 \
+  --cudagraph-capture-sizes 1 2 4 8 12 16 20 24 28 32 \
+  --dcp-kv-cache-interleave-size 4 \
+  --decode-context-parallel-size 2 \
+  --default-chat-template-kwargs '{"reasoning_effort":"max","clear_thinking":false}' \
+  --dtype bfloat16 \
+  --enable-auto-tool-choice \
+  --enable-chunked-prefill \
+  --no-enable-flashinfer-autotune \
+  --enable-prefix-caching \
+  --enable-prompt-tokens-details \
+  --gpu-memory-utilization 0.985 \
+  --kv-cache-dtype fp8 \
+  --kv-cache-memory-bytes 7650410496 \
+  --linear-backend b12x \
+  --load-format instanttensor \
+  --mamba-cache-mode align \
+  --max-cudagraph-capture-size 32 \
+  --max-model-len 1048576 \
+  --max-num-batched-tokens 4096 \
+  --max-num-seqs 8 \
+  --max-parallel-prefills 1 \
+  --moe-backend b12x \
+  --override-generation-config '{"temperature":1.0,"top_p":0.95}' \
+  --prefill-compute-share 0.4 \
+  --prefill-schedule-interval 1 \
+  --quantization modelopt_mixed \
+  --reasoning-parser glm45 \
+  --recurrent-checkpoint-policy request_boundaries \
+  --speculative-config '{"method":"mtp","draft_sample_method":"probabilistic","rejection_sample_method":"standard","moe_backend":"b12x","attention_backend":"B12X","num_speculative_tokens":3}' \
+  --tensor-parallel-size 2 \
+  --tool-call-parser glm47
+```
+
+## Ethical Considerations
+
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+
+Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.
+
+For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards below.
+
+Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
+
+SUBCARDS:
+## Explainability
+
+
+| Field                                    | Response                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+|:----------------------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
+| **Intended Task/Domain:**                | Text generation, reasoning, summarization, and question answering.                                                                                                                                                                                                                                                                                                                                                                                       |
+| **Model Type:**                          | Text, Image, and Video-to-text transformer                                                                                                                                                                                                                                                                                                                                                                                                               |
+| **Intended Users:**                      | This model is intended for developers, researchers, and customers building/utilizing LLMs, while balancing accuracy and efficiency.                                                                                                                                                                                                                                                                                                                     |
+| **Output:**                              | Text String(s)                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+| **Describe how the model works:**        | Generates text by predicting the next word or token based on the context provided in the input sequence using multiple self-attention layers.                                                                                                                                                                                                                                                                                                            |
+| **Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of:**        | Not Applicable    |
+| **Technical Limitations & Mitigation:**  | The model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. Therefore, before deploying any applications of this model, developers should perform safety testing and tuning tailored to their specific applications of the model.                                       |
+| **Verified to have met prescribed NVIDIA quality standards:** | Yes                                                                                                                                                                                                                                                                                                                                                                                                                                |
+| **Performance Metrics:**                 | Accuracy, Throughput, and user-side throughput                                                                                                                                                                                                                                                                                                                                                                                                           |
+| **Potential Known Risk**                 | The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.                                                                                                                                                                                                       |
+| **Licensing:**                           | **GOVERNING DOWNLOAD TERMS:** Use of this model is governed by [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/).
+
+**ADDITIONAL INFORMATION** : The GLM-5.3-Flash base model is governed by the [MIT License](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) . **GLM-5.3-Flash**).                                                                                                                           |
+
+
+## Bias
+
+| Field                                                                                                           | Response |
+|:---------------------------------------------------------------------------------------------------------------:|:--------:|
+| Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/protected-classes) in model design and testing:       | None     |
+| Measures taken to mitigate against unwanted bias                                                                | None     |
+
+
+## Safety & Security
+
+| Field                              | Response                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+|:-----------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
+| Model Application Field(s):         | Chat, Instruction Following, Chatbot Development, Code Generation, Reasoning                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+| Describe the life critical impact (if present) | Not Applicable                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+| Use Case Restrictions:              | Abide by the **GOVERNING DOWNLOAD TERMS:** Use of this model is governed by [NVIDIA Open Model Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/).
+
+**ADDITIONAL INFORMATION** : The GLM-5.3-Flash base model is governed by the [MIT License](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) . **GLM-5.3-Flash**).                                                                                                                                                                                                                   |
+| Model and Dataset Restrictions:     | The Principle of least privilege (PoLP) is applied limiting access for dataset generation. Restrictions enforce dataset access during training, and dataset license constraints adhered to. Model checkpoints are made available on Hugging Face, and may become available on cloud providers' model catalog.                                                                                                                                                                                                                                   |
+
+
+## Privacy
+
+
+| Field                                                                          | Response                                                                           |
+| ------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------- |
+| **Generatable or reverse-engineerable personal data**                          | No 
+| **Personal data used to create this model**                                    | None Known                                                                         |      |
+| **Was consent obtained for any personal data used**                            | Not Applicable                                                                     |
+| **How often is the dataset reviewed**                                          | Before Release                                                                     |
+| **Was data from user interactions with the AI model used to train the model**  | No                                                                                 |
+| **Is there provenance for all datasets used in training**                      | Yes                                                                                |
+| **Does data labeling comply with privacy laws**                                | Yes                                                                                |
+| **Is the data compliant with data-subject requests for correction or removal** | No, not possible with externally-sourced data.                                     |
+| **Applicable NVIDIA Privacy Policy**                                           | [NVIDIA Privacy Policy](https://www.nvidia.com/en-us/about-nvidia/privacy-policy/) |                 |

chat_template.jinja

@@ -0,0 +1,261 @@
+[gMASK]<sop>
+{%- set effective_reasoning_effort = reasoning_effort if reasoning_effort is defined and reasoning_effort in ['low', 'high'] else 'max' -%}
+{%- if effective_reasoning_effort is not none -%}<|system|>Reasoning Effort: {{ effective_reasoning_effort | capitalize }}{%- endif -%}
+{%- set clear_thinking = clear_thinking if clear_thinking is defined else false -%}
+{%- if tools -%}
+{%- macro tool_to_json(tool) -%}
+    {%- set ns_tool = namespace(first=true) -%}
+    {{ '{' -}}
+    {%- for k, v in tool.items() -%}
+        {%- if k != 'defer_loading' and k != 'strict' -%}
+            {%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
+            {%- set ns_tool.first = false -%}
+            "{{ k }}": {{ v | tojson(ensure_ascii=False) }}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- '}' -}}
+{%- endmacro -%}
+{%- macro tool_references_to_response(refs) -%}
+    {{- '<tool_response><tools>\n' -}}
+    {%- for tr in refs -%}
+        {%- for tool in tools -%}
+            {%- if 'function' in tool -%}
+                {%- set tool = tool['function'] -%}
+            {%- endif -%}
+            {%- if tool.name == tr.name -%}
+                {{- tool_to_json(tool) + '\n' -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endfor -%}
+    {{- '</tools></tool_response>' -}}
+{%- endmacro -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{%- if 'function' in tool -%}
+    {%- set tool = tool['function'] -%}
+{%- endif -%}
+{% if tool.defer_loading is not defined or not tool.defer_loading %}
+{{ tool_to_json(tool) }}
+{% endif %}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
+{%- macro emit_image() -%}<|begin_of_image|><|image|><|end_of_image|>{%- endmacro -%}
+{%- macro emit_video() -%}<|begin_of_video|><|video|><|end_of_video|>{%- endmacro -%}
+{%- macro emit_audio() -%}<|begin_of_audio|><|end_of_audio|>{%- endmacro -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content -}}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text -}}
+            {%- elif item is string -%}
+                {{- item -}}
+            {%- elif item is mapping and item.type in ['image', 'image_url'] -%}
+                {{- emit_image() -}}
+            {%- elif item is mapping and item.type in ['video', 'video_url'] -%}
+                {{- emit_video() -}}
+            {%- elif item is mapping and item.type in ['audio', 'audio_url', 'input_audio'] -%}
+                {{- emit_audio() -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- elif content is not none -%}
+        {{- content -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro tool_response(text) -%}
+{{- '<tool_response>' + text + '</tool_response>' -}}
+{%- endmacro -%}
+{%- macro render_tool_response(m) -%}
+{%- if m.content is string -%}
+    {{- tool_response(m.content) -}}
+{%- elif m.content and m.content is not mapping and m.content.0.type == "tool_reference" -%}
+    {{- tool_references_to_response(m.content) -}}
+{%- elif is_list_of_outputs(m) -%}
+    {%- for tr in m.content -%}
+        {%- if tr.output is iterable and tr.output is not string and tr.output is not mapping and tr.output and tr.output.0.type == "tool_reference" -%}
+            {{- tool_references_to_response(tr.output) -}}
+        {%- else -%}
+            {{- tool_response(visible_text(tr.output)) -}}
+        {%- endif -%}
+    {%- endfor -%}
+{%- else -%}
+    {{- tool_response(visible_text(m.content)) -}}
+{%- endif -%}
+{%- endmacro -%}
+{%- macro id_of(obj) -%}
+    {%- if obj.tool_call_id -%}
+        {{- obj.tool_call_id -}}
+    {%- elif obj.id -%}
+        {{- obj.id -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro is_list_of_outputs(m) -%}
+    {%- if m.content and m.content.0.output is defined -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro has_dup_tool_result_id(lo, hi, target) -%}
+    {%- set ns_cnt = namespace(n=0) -%}
+    {%- for k in range(lo, hi + 1) -%}
+        {%- set m = messages[k] -%}
+        {%- if is_list_of_outputs(m) -%}
+            {%- for entry in m.content -%}
+                {%- if id_of(entry) == target -%}
+                    {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- elif id_of(m) == target -%}
+            {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+        {%- endif -%}
+        {%- if ns_cnt.n > 1 -%}{%- break -%}{%- endif -%}
+    {%- endfor -%}
+    {%- if ns_cnt.n > 1 -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro tc_id_exists(tcs, target) -%}
+    {%- set ns_f = namespace(found=false) -%}
+    {%- for tc in tcs -%}
+        {%- if id_of(tc) == target -%}
+            {%- set ns_f.found = true -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- if ns_f.found -%}1{%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) -%}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{%- for m in messages -%}
+{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- elif '</think>' in content %}
+    {%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
+    {%- set content = content.split('</think>')[-1] %}
+{%- endif %}
+{%- if (not clear_thinking or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
+{{ '<think>' + reasoning_content +  '</think>'}}
+{%- else -%}
+{{ '<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{- '<tool_call>' ~ tc.name -}}
+{% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' -}}
+    {%- set block_start = loop.index0 -%}
+    {%- set ns_blk = namespace(end=block_start) -%}
+    {%- for j in range(block_start, messages|length) -%}
+        {%- if messages[j].role == 'tool' -%}
+            {%- set ns_blk.end = j -%}
+        {%- else -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns_a = namespace(tool_calls=none) -%}
+    {%- if block_start > 0 and messages[block_start - 1].role == 'assistant' and messages[block_start - 1].tool_calls -%}
+        {%- set ns_a.tool_calls = messages[block_start - 1].tool_calls -%}
+    {%- endif -%}
+    {%- set ns_chk = namespace(can_sort=true) -%}
+    {%- if not ns_a.tool_calls -%}
+        {%- set ns_chk.can_sort = false -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {%- if not ns_chk.can_sort -%}{%- break -%}{%- endif -%}
+            {%- set m = messages[k] -%}
+            {%- if is_list_of_outputs(m) -%}
+                {%- for entry in m.content -%}
+                    {%- if not ns_chk.can_sort -%}{%- break -%}{%- endif -%}
+                    {%- set eid = id_of(entry) -%}
+                    {%- if not eid -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif has_dup_tool_result_id(block_start, ns_blk.end, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif not tc_id_exists(ns_a.tool_calls, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- else -%}
+                {%- set tk_id = id_of(m) -%}
+                {%- if not tk_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif has_dup_tool_result_id(block_start, ns_blk.end, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif not tc_id_exists(ns_a.tool_calls, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- for i in range(ns_a.tool_calls | length) -%}
+            {%- if not ns_chk.can_sort -%}{%- break -%}{%- endif -%}
+            {%- set tc_id = id_of(ns_a.tool_calls[i]) -%}
+            {%- if not tc_id -%}
+                {%- set ns_chk.can_sort = false -%}
+            {%- endif -%}
+            {%- for j in range(i + 1, ns_a.tool_calls | length) -%}
+                {%- if id_of(ns_a.tool_calls[j]) == tc_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                    {%- break -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- if ns_chk.can_sort -%}
+        {%- for tc in ns_a.tool_calls -%}
+            {%- set tc_id = id_of(tc) -%}
+            {%- for k in range(block_start, ns_blk.end + 1) -%}
+                {%- set m = messages[k] -%}
+                {%- if is_list_of_outputs(m) -%}
+                    {%- for entry in m.content -%}
+                        {%- set eid = id_of(entry) -%}
+                        {%- if eid == tc_id -%}
+                            {%- if entry.output is iterable and entry.output is not string and entry.output is not mapping and entry.output and entry.output.0.type == "tool_reference" -%}
+                                {{- tool_references_to_response(entry.output) -}}
+                            {%- else -%}
+                                {{- tool_response(visible_text(entry.output)) -}}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                {%- else -%}
+                    {%- set tk_id = id_of(m) -%}
+                    {%- if tk_id == tc_id -%}
+                        {{- render_tool_response(m) -}}
+                    {%- endif -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endfor -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {{- render_tool_response(messages[k]) -}}
+        {%- endfor -%}
+    {%- endif -%}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '<think>' -}}
+{%- endif -%}

config.json

@@ -0,0 +1,3263 @@
+{
+  "architectures": [
+    "Glm5NextForConditionalGeneration"
+  ],
+  "image_end_token_id": 154831,
+  "image_start_token_id": 154830,
+  "image_token_id": 154854,
+  "model_type": "glm5_next",
+  "quantization_config": {
+    "exclude_modules": [],
+    "quant_algo": "MIXED_PRECISION",
+    "quant_method": "modelopt",
+    "quantized_layers": {
+      "lm_head": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.0.mlp.down_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.0.mlp.gate_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.0.mlp.up_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.0.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.0.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.0.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.0.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.1.mlp.down_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.1.mlp.gate_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.1.mlp.up_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.1.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.1.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.1.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.1.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.10.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.10.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.10.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.10.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.10.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.10.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.10.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.10.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.11.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.11.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.11.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.11.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.11.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.11.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.11.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.11.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.12.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.12.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.12.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.12.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.12.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.12.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.12.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.12.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.13.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.13.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.13.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.13.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.13.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.13.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.13.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.13.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.14.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.14.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.14.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.14.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.14.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.14.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.14.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.14.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.15.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.15.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.15.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.15.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.15.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.15.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.15.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.15.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.16.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.16.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.16.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.16.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.16.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.16.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.16.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.16.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.17.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.17.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.17.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.17.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.17.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.17.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.17.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.17.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.18.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.18.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.18.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.18.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.18.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.18.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.18.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.18.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.19.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.19.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.19.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.19.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.19.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.19.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.19.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.19.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.2.mlp.down_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.2.mlp.gate_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.2.mlp.up_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.2.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.2.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.2.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.2.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.20.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.20.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.20.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.20.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.20.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.20.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.20.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.20.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.21.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.21.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.21.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.21.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.21.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.21.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.21.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.21.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.22.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.22.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.22.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.22.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.22.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.22.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.22.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.22.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.23.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.23.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.23.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.23.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.23.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.23.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.23.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.23.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.24.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.24.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.24.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.24.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.24.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.24.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.24.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.24.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.25.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.25.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.25.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.25.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.25.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.25.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.25.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.25.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.26.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.26.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.26.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.26.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.26.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.26.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.26.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.26.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.27.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.27.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.27.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.27.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.27.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.27.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.27.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.27.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.28.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.28.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.28.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.28.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.28.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.28.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.28.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.28.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.29.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.29.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.29.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.29.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.29.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.29.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.29.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.29.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.3.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.3.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.3.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.3.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.3.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.3.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.3.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.3.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.30.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.30.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.30.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.30.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.30.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.30.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.30.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.30.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.31.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.31.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.31.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.31.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.31.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.31.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.31.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.31.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.32.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.32.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.32.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.32.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.32.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.32.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.32.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.32.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.33.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.33.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.33.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.33.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.33.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.33.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.33.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.33.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.34.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.34.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.34.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.34.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.34.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.34.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.34.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.34.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.35.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.35.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.35.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.35.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.35.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.35.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.35.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.35.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.36.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.36.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.36.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.36.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.36.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.36.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.36.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.36.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.37.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.37.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.37.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.37.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.37.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.37.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.37.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.37.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.38.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.38.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.38.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.38.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.38.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.38.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.38.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.38.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.39.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.39.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.39.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.39.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.39.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.39.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.39.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.39.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.4.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.4.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.4.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.4.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.4.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.4.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.4.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.4.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.40.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.40.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.40.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.40.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.40.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.40.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.40.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.40.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.41.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.41.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.41.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.41.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.41.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.41.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.41.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.41.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.42.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.42.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.42.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.42.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.42.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.42.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.42.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.42.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.43.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.43.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.43.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.43.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.43.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.43.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.43.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.43.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.44.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.44.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.44.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.44.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.44.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.44.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.44.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.44.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.45.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.language_model.layers.45.mlp.shared_experts.down_proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.mlp.shared_experts.gate_proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.mlp.shared_experts.up_proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.self_attn.indexer.weights_proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.self_attn.indexer.wk": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.self_attn.indexer.wq_b": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.self_attn.kv_a_proj_with_mqa": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.self_attn.kv_b_proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.self_attn.o_proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.self_attn.q_a_proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.45.self_attn.q_b_proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.language_model.layers.5.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.5.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.5.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.5.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.5.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.5.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.5.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.5.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.6.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.6.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.6.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.6.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.6.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.6.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.6.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.6.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.7.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.7.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.7.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.7.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.7.self_attn.kv_a_proj_with_mqa": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.7.self_attn.o_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.7.self_attn.q_a_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.7.self_attn.q_b_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.8.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.8.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.8.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.8.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.8.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.8.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.8.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.8.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.9.mlp.experts": {
+        "group_size": 16,
+        "quant_algo": "NVFP4",
+        "weight_scale_encoding": "csf_32x2_v1"
+      },
+      "model.language_model.layers.9.mlp.shared_experts.down_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.9.mlp.shared_experts.gate_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.9.mlp.shared_experts.up_proj": {
+        "quant_algo": "FP8_PB_WO",
+        "weight_block_size": [
+          128,
+          128
+        ]
+      },
+      "model.language_model.layers.9.self_attn.k_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.9.self_attn.o_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.9.self_attn.q_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.language_model.layers.9.self_attn.v_proj": {
+        "quant_algo": "MXFP8",
+        "weight_block_size": [
+          1,
+          32
+        ]
+      },
+      "model.visual.blocks.0.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.0.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.0.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.0.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.0.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.1.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.1.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.1.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.1.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.1.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.10.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.10.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.10.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.10.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.10.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.11.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.11.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.11.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.11.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.11.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.12.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.12.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.12.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.12.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.12.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.13.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.13.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.13.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.13.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.13.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.14.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.14.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.14.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.14.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.14.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.15.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.15.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.15.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.15.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.15.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.16.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.16.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.16.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.16.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.16.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.17.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.17.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.17.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.17.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.17.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.18.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.18.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.18.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.18.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.18.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.19.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.19.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.19.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.19.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.19.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.2.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.2.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.2.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.2.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.2.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.20.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.20.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.20.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.20.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.20.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.21.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.21.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.21.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.21.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.21.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.22.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.22.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.22.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.22.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.22.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.23.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.23.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.23.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.23.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.23.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.3.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.3.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.3.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.3.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.3.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.4.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.4.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.4.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.4.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.4.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.5.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.5.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.5.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.5.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.5.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.6.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.6.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.6.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.6.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.6.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.7.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.7.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.7.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.7.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.7.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.8.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.8.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.8.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.8.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.8.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.9.attn.proj": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.9.attn.qkv": {
+        "group_size": 32,
+        "quant_algo": "MXFP8"
+      },
+      "model.visual.blocks.9.mlp.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.9.mlp.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.blocks.9.mlp.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.merger.down_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.merger.gate_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.merger.proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      },
+      "model.visual.merger.up_proj": {
+        "group_size": 16,
+        "quant_algo": "W4A16_NVFP4"
+      }
+    }
+  },
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "eos_token_id": [
+      154820,
+      154827,
+      154829
+    ],
+    "first_k_dense_replace": 3,
+    "hc_eps": 1e-06,
+    "hc_mult": 4,
+    "hc_sinkhorn_iters": 20,
+    "head_dim": 0,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "index_head_dim": 128,
+    "index_kpool": 4,
+    "index_kpool_always_select_tail": true,
+    "index_kpool_compress": true,
+    "index_n_heads": 32,
+    "index_share_for_mtp_iteration": true,
+    "index_topk": 2048,
+    "indexer_rope_interleave": true,
+    "indexer_types": [
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full"
+    ],
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "kv_lora_rank": 512,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention"
+    ],
+    "linear_attn_config": {
+      "full_attn_layers": [
+        3,
+        7,
+        11,
+        15,
+        19,
+        23,
+        27,
+        31,
+        35,
+        39,
+        43
+      ],
+      "gate_lower_bound": -5.0,
+      "head_dim": 128,
+      "kda_layers": [
+        0,
+        1,
+        2,
+        4,
+        5,
+        6,
+        8,
+        9,
+        10,
+        12,
+        13,
+        14,
+        16,
+        17,
+        18,
+        20,
+        21,
+        22,
+        24,
+        25,
+        26,
+        28,
+        29,
+        30,
+        32,
+        33,
+        34,
+        36,
+        37,
+        38,
+        40,
+        41,
+        42,
+        44
+      ],
+      "num_heads": 64,
+      "short_conv_kernel_size": 4
+    },
+    "max_position_embeddings": 1048576,
+    "mhc": true,
+    "mla_use_nope": true,
+    "mlp_layer_types": [
+      "dense",
+      "dense",
+      "dense",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse"
+    ],
+    "model_type": "glm5_next_text",
+    "moe_intermediate_size": 2048,
+    "moe_router_dtype": "float32",
+    "n_group": 1,
+    "n_routed_experts": 288,
+    "n_shared_experts": 1,
+    "norm_topk_prob": true,
+    "num_attention_heads": 64,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 45,
+    "num_key_value_heads": 64,
+    "num_nextn_predict_layers": 1,
+    "output_router_logits": false,
+    "pad_token_id": 154820,
+    "q_lora_rank": 1536,
+    "qk_head_dim": 256,
+    "qk_nope_head_dim": 256,
+    "qk_rope_head_dim": 0,
+    "rms_norm_eps": 1e-05,
+    "routed_scaling_factor": 2.5,
+    "router_aux_loss_coef": 0.001,
+    "scoring_func": "sigmoid",
+    "swiglu_limit": 10.0,
+    "tie_word_embeddings": false,
+    "topk_group": 1,
+    "topk_method": "noaux_tc",
+    "use_cache": true,
+    "v_head_dim": 256,
+    "vocab_size": 154880
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.16.0",
+  "video_end_token_id": 154833,
+  "video_start_token_id": 154832,
+  "video_token_id": 154855,
+  "vision_config": {
+    "attention_bias": true,
+    "attention_dropout": 0.0,
+    "depth": 24,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "image_size": 448,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "glm5_next_vision",
+    "num_heads": 16,
+    "out_hidden_size": 4096,
+    "patch_size": 14,
+    "projection_intermediate_size": 10240,
+    "rms_norm_eps": 1e-05,
+    "spatial_merge_size": 2,
+    "swiglu_limit": 10.0,
+    "temporal_patch_size": 2
+  }
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "pad_token_id": 154820,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "transformers_version": "5.16.0"
+}

hf_quant_config.json

@@ -0,0 +1,2973 @@
+{
+  "exclude_modules": [],
+  "quant_algo": "MIXED_PRECISION",
+  "quant_method": "modelopt",
+  "quantized_layers": {
+    "lm_head": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.0.mlp.down_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.0.mlp.gate_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.0.mlp.up_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.0.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.0.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.0.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.0.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.1.mlp.down_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.1.mlp.gate_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.1.mlp.up_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.1.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.1.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.1.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.1.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.10.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.10.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.10.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.10.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.10.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.10.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.10.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.10.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.11.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.11.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.11.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.11.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.11.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.11.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.11.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.11.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.12.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.12.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.12.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.12.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.12.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.12.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.12.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.12.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.13.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.13.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.13.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.13.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.13.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.13.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.13.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.13.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.14.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.14.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.14.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.14.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.14.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.14.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.14.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.14.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.15.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.15.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.15.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.15.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.15.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.15.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.15.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.15.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.16.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.16.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.16.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.16.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.16.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.16.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.16.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.16.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.17.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.17.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.17.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.17.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.17.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.17.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.17.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.17.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.18.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.18.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.18.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.18.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.18.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.18.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.18.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.18.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.19.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.19.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.19.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.19.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.19.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.19.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.19.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.19.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.2.mlp.down_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.2.mlp.gate_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.2.mlp.up_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.2.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.2.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.2.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.2.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.20.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.20.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.20.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.20.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.20.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.20.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.20.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.20.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.21.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.21.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.21.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.21.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.21.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.21.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.21.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.21.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.22.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.22.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.22.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.22.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.22.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.22.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.22.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.22.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.23.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.23.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.23.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.23.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.23.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.23.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.23.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.23.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.24.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.24.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.24.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.24.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.24.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.24.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.24.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.24.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.25.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.25.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.25.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.25.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.25.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.25.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.25.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.25.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.26.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.26.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.26.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.26.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.26.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.26.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.26.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.26.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.27.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.27.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.27.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.27.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.27.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.27.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.27.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.27.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.28.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.28.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.28.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.28.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.28.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.28.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.28.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.28.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.29.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.29.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.29.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.29.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.29.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.29.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.29.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.29.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.3.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.3.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.3.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.3.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.3.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.3.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.3.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.3.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.30.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.30.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.30.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.30.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.30.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.30.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.30.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.30.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.31.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.31.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.31.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.31.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.31.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.31.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.31.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.31.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.32.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.32.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.32.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.32.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.32.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.32.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.32.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.32.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.33.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.33.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.33.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.33.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.33.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.33.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.33.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.33.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.34.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.34.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.34.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.34.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.34.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.34.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.34.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.34.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.35.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.35.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.35.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.35.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.35.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.35.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.35.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.35.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.36.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.36.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.36.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.36.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.36.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.36.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.36.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.36.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.37.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.37.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.37.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.37.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.37.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.37.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.37.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.37.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.38.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.38.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.38.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.38.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.38.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.38.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.38.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.38.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.39.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.39.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.39.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.39.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.39.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.39.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.39.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.39.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.4.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.4.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.4.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.4.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.4.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.4.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.4.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.4.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.40.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.40.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.40.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.40.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.40.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.40.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.40.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.40.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.41.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.41.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.41.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.41.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.41.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.41.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.41.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.41.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.42.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.42.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.42.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.42.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.42.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.42.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.42.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.42.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.43.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.43.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.43.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.43.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.43.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.43.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.43.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.43.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.44.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.44.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.44.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.44.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.44.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.44.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.44.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.44.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.45.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.language_model.layers.45.mlp.shared_experts.down_proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.mlp.shared_experts.gate_proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.mlp.shared_experts.up_proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.self_attn.indexer.weights_proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.self_attn.indexer.wk": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.self_attn.indexer.wq_b": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.self_attn.kv_a_proj_with_mqa": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.self_attn.kv_b_proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.self_attn.o_proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.self_attn.q_a_proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.45.self_attn.q_b_proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.language_model.layers.5.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.5.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.5.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.5.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.5.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.5.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.5.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.5.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.6.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.6.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.6.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.6.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.6.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.6.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.6.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.6.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.7.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.7.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.7.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.7.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.7.self_attn.kv_a_proj_with_mqa": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.7.self_attn.o_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.7.self_attn.q_a_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.7.self_attn.q_b_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.8.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.8.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.8.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.8.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.8.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.8.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.8.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.8.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.9.mlp.experts": {
+      "group_size": 16,
+      "quant_algo": "NVFP4",
+      "weight_scale_encoding": "csf_32x2_v1"
+    },
+    "model.language_model.layers.9.mlp.shared_experts.down_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.9.mlp.shared_experts.gate_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.9.mlp.shared_experts.up_proj": {
+      "quant_algo": "FP8_PB_WO",
+      "weight_block_size": [
+        128,
+        128
+      ]
+    },
+    "model.language_model.layers.9.self_attn.k_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.9.self_attn.o_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.9.self_attn.q_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.language_model.layers.9.self_attn.v_proj": {
+      "quant_algo": "MXFP8",
+      "weight_block_size": [
+        1,
+        32
+      ]
+    },
+    "model.visual.blocks.0.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.0.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.0.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.0.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.0.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.1.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.1.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.1.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.1.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.1.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.10.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.10.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.10.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.10.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.10.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.11.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.11.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.11.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.11.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.11.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.12.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.12.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.12.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.12.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.12.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.13.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.13.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.13.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.13.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.13.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.14.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.14.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.14.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.14.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.14.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.15.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.15.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.15.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.15.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.15.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.16.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.16.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.16.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.16.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.16.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.17.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.17.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.17.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.17.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.17.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.18.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.18.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.18.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.18.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.18.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.19.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.19.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.19.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.19.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.19.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.2.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.2.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.2.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.2.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.2.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.20.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.20.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.20.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.20.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.20.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.21.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.21.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.21.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.21.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.21.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.22.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.22.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.22.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.22.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.22.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.23.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.23.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.23.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.23.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.23.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.3.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.3.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.3.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.3.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.3.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.4.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.4.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.4.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.4.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.4.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.5.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.5.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.5.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.5.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.5.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.6.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.6.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.6.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.6.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.6.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.7.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.7.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.7.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.7.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.7.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.8.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.8.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.8.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.8.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.8.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.9.attn.proj": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.9.attn.qkv": {
+      "group_size": 32,
+      "quant_algo": "MXFP8"
+    },
+    "model.visual.blocks.9.mlp.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.9.mlp.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.blocks.9.mlp.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.merger.down_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.merger.gate_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.merger.proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    },
+    "model.visual.merger.up_proj": {
+      "group_size": 16,
+      "quant_algo": "W4A16_NVFP4"
+    }
+  }
+}

model-00001-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d2fcce2097ab08926290ff099b792c688a9eb014a6963ba201e6f6c18767730
+size 4294380516

model-00002-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abe5800514c86b4fe57a9f8419a1343460c5459495f626a15c143c077f0eed59
+size 398331164

model-00003-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6bf622464604faa5e7d8f3c7c265242fc768f90f5069bc5bff84d5665c3722f
+size 4187354632

model-00004-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bf321c5b0c6a866f1916587cb84e0f67b6527fc9da1f432ce8ddb1eeefaf89f
+size 4059621056

model-00005-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa93cdc2b83135bfd48c38a88e5d9299132e8b0d919ed2e68343fce8c7f1c844
+size 1204588600

model-00006-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f36a0aebce974e605a66c07c10963671753823640e8d54584a98fd04578414a
+size 3386693718

model-00007-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08cf34e007f01060b73d63b9e73e3fb7429aef998e61bb521d927a5e1c836ca4
+size 3386630980

model-00008-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27665caafba3db19034967c9f6c91e9614c0b71270a2a5983131ff845b34c66a
+size 3386784380

model-00009-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fefef9818a88957a172129075db03351ff8705f912d187439650b548e882d1f
+size 3386714856

model-00010-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:618704500d946c3c573e9eca8ca74f0395139e190a76687feff6930201c32c02
+size 3386642180

model-00011-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e79fd0c2161ffd83b02aba108713a0864959e26cb36cedaca5c1fecd1328d29b
+size 3386705796

model-00012-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc3ce32db330f4edaed1a814da5709e816b2b5874c582124cb2a319c123f0fb5
+size 3386704188

model-00013-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e44d1253f3fa790150f3c869c3fd04d692c08fcf29f54e9ed0f86f1e3804f44e
+size 3386634084

model-00014-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9242f5c32541c58f9e9850091664f7359c6d4536254c5f808b68f133487404f
+size 3386785778

model-00015-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:856b041abe136f7bdf624df0e3f816999020e0ceaa44339f8f914cfe6423686c
+size 5080037446

model-00016-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4cc6b1eca1fd60ef05f4ed8b58fc31fc852fde068544fc1e06badcfae51e9dba
+size 5080062512

model-00017-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2dc2ef526eccd2dc1df753aa6c22529a12340270b241f6429d915d43c8ffaf37
+size 5080033556

model-00018-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1fa1a86341b82892fb5ed234ad82d7836d3fcf004675a000e366116a423144c
+size 5080002106

model-00019-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49ed0fd14feeb265c8c80e68b007feb6612f220b17cd7bec0382cf4fccdedad3
+size 5080044836

model-00020-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36d13ead5eff2bebc082ea8bfff2cd7952249e3d464e4691f5b33a7334095da8
+size 5080015448

model-00021-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be455e23d8170063b8ac2d22be604e71594fe72755c36b31272253cf8f94447c
+size 5080033720

model-00022-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f7a0f6aeea5a2ff95b7c2f993675eca757cb3bd7e0ed9876850f73c0c640393
+size 5080007626

model-00023-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18d7db72e3737476c604f6f23615318158e91026226eafcc3975d23293e36bbe
+size 5080036888

model-00024-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:937b0331965662e4d53e7266006bf9268dcf6e511354486396eb2e1cd1e1fce5
+size 5079998124

model-00025-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:522b327ece804e547f601a266be74cbdd4dfbc2938419052d43a5e1283ba94f5
+size 5080015080

model-00026-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa5956b9758a382ff3a63a5c4625dd2a876183a8f6083bac1b665a7eb93903e1
+size 5080002452

model-00027-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e2b7685470b601aceebe226b26b52464fa40d74b9b346af337d45e8faacc1cc
+size 5080025960

model-00028-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83df1ae7d43eaf70b29fec3194851cb924775b8671440ed02ff2ac066cbaf7e5
+size 5080010694

model-00029-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:223f763f1a1569d664f3e00641043bd6a0a0fe6c7881126620ef0183235fd5cd
+size 5080031820

model-00030-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36e81695f6ddab8108aeb7479b7015a21b8ddb9057b4375527c13fb29af38392
+size 5080122474

model-00031-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:697493ca2ce3f7fc975aaf9d8e65a2b373a2b4331035a4f5379ee4904688c73f
+size 5080041150

model-00032-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1967e72e60ed152b39ab0d951a3c524a885c6c2fb760e55688a08ccf197ae30b
+size 5080082340

model-00033-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3896f4b763903405bfdf37249296bb05f525ca2134541dab7ea91c5513047a1c
+size 5080023370

model-00034-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50d2cd8d608d3b8c59bd4c654275d6f840cd7598aeb42a2a5da8db75ae5c8b4d
+size 5079989500

model-00035-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27ba1ee0f211a331fff5221ffe5f00b06eeb002f1865ae8d4fd8335d2c7f4c8c
+size 5080032548

model-00036-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6954778ce40d39efce3a07c01e1d245e757a701d72ead7a73e48fc2a57cbddd
+size 5080044714

model-00037-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a956ea3bd9468184565c5e94211c663bd401b3c1c57da08e96d5fd2b49419d9
+size 5080045746

model-00038-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c12da94d3c1642f5ec34d1c1f573d193c75818bfc13b4774954cbd1ec9bfb646
+size 5080051898

model-00039-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5262d01f162e665c5da0f4e841d450e2396b4e45727b704c0ac1e514e287e7a9
+size 5080035462

model-00040-of-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73342fa305b49ec6677423b090479bbd8d021cd7e9f0afc27c2eeaa0b678dc23
+size 5080022666

model-inputscales.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3dd6c79b081f1b61bd13a9354a0753f5b0406e3a4e8de05c45ff5c74e56ef1b
+size 4726696

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1981941b2fabb9271ae3a3cb76be6ea41927f730b6633471d3c147106703aeea
+size 35266806

processor_config.json

@@ -0,0 +1,44 @@
+{
+  "image_processor": {
+    "do_rescale": true,
+    "patch_expand_factor": 1,
+    "merge_size": 2,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "temporal_patch_size": 2,
+    "patch_size": 14,
+    "min_image_tokens": 16,
+    "max_image_tokens": 8000,
+    "image_processor_type": "Glm5NextImageProcessor"
+  },
+  "video_processor": {
+    "do_rescale": true,
+    "video_processor_type": "Glm5NextVideoProcessor",
+    "patch_expand_factor": 1,
+    "merge_size": 2,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "temporal_patch_size": 2,
+    "patch_size": 14,
+    "min_image_tokens": 16,
+    "max_image_tokens": 240000,
+    "fps": 2
+  },
+  "processor_class": "Glm5NextProcessor"
+}

tensors/native-QB.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3911ecc99b3a22a882b6cd892e7d948f099d069e61915dcfaa67b2cce00ffd41
+size 753692264