Xiao-AMD commited on
Commit
0d69595
·
verified ·
1 Parent(s): 3966522

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -15,10 +15,9 @@ base_model:
15
  - **Operating System(s):** Linux
16
  - **Inference Engine:** [vLLM](https://docs.vllm.ai/en/latest/)
17
  - **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (V0.12)
18
- - **Weight quantization:** MoE/Linear, OCP MXFP4 (per-group, group size 32), Static
19
- - **Activation quantization:** OCP MXFP4 (per-group, group size 32), Dynamic
20
- - **Attention quantization:** FP8 (E4M3), per-channel
21
- - **Excluded from quantization:** `self_attn` weights kept at FP8, MoE gates, `lm_head`, vision tower and multimodal projector kept at BF16
22
 
23
  This model was built with the Kimi-K2.7-Code model by applying [AMD-Quark](https://quark.docs.amd.com/latest/index.html) for MXFP4 quantization.
24
 
 
15
  - **Operating System(s):** Linux
16
  - **Inference Engine:** [vLLM](https://docs.vllm.ai/en/latest/)
17
  - **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (V0.12)
18
+ - **Weight quantization:** OCP MXFP4, Static; self_attn Perchannel, FP8E4M3, Static
19
+ - **Activation quantization:** OCP MXFP4, Dynamic; self_attn FP8E4M3, per-channel, Dynamic
20
+ - **Excluded from quantization:** MoE gates, `lm_head`, vision tower and multimodal projector
 
21
 
22
  This model was built with the Kimi-K2.7-Code model by applying [AMD-Quark](https://quark.docs.amd.com/latest/index.html) for MXFP4 quantization.
23