| --- |
| base_model: |
| - MiniMaxAI/MiniMax-M3.1-Flash |
| license: other |
| license_name: minimax-community |
| license_link: LICENSE |
| --- |
| |
| # Model Overview |
|
|
| - **Model Architecture:** MiniMaxM3SparseForConditionalGeneration |
| - **Input:** Text, Image, Video |
| - **Output:** Text |
| - **Supported Hardware Microarchitecture:** AMD MI350/MI355 |
| - **ROCm**: 7.2.3 |
| - **PyTorch**: 2.13.0 |
| - **Transformers**: 5.19.0 |
| - **Operating System(s):** Linux |
| - **Inference Engine:** vLLM, SGLang |
| - **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) |
| - **Routed-expert weight quantization:** OCP MXFP4, Static |
| - **Routed-expert activation quantization:** OCP MXFP4, Dynamic |
| - **All other layers**: unchanged precision compared to base [MiniMaxAI/MiniMax-M3.1-Flash](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash) |
|
|
| # Model serving |
|
|
| Please refer to [vLLM recipe](https://recipes.vllm.ai/) and [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1). |
|
|
|
|
| Example usage: |
|
|
| ## vLLM |
|
|
| ```bash |
| VLLM_ROCM_USE_AITER=1 \ |
| vllm serve amd/MiniMax-M3.1-Flash-Quark-MXFP4 \ |
| --tensor-parallel-size 4 \ |
| --reasoning-parser minimax_m3 \ |
| --block-size 128 \ |
| --speculative-config '{"method":"dspark","model":"amd/MiniMax-M3.1-Flash-Quark-MXFP4/dspark","num_speculative_tokens":7,"attention_backend":"FLASHINFER_TS","kv_cache_dtype":"fp8","draft_sample_method":"probabilistic","rejection_sample_method":"block"}' \ |
| --disable-hybrid-kv-cache-manager" |
| ``` |
|
|
| ## SGLang |
|
|
| ```bash |
| SGLANG_USE_AITER=1 \ |
| sglang serve \ |
| --trust-remote-code \ |
| --model-path amd/MiniMax-M3.1-Flash-Quark-MXFP4 \ |
| --reasoning-parser minimax-m3 \ |
| --tool-call-parser minimax-m3 \ |
| --tp 4 \ |
| --speculative-algorithm DSPARK \ |
| --speculative-use-block-verification \ |
| --cuda-graph-backend-prefill breakable \ |
| --fp8-gemm-backend aiter \ |
| --chunked-prefill-size 32768 \ |
| --max-running-requests 128 \ |
| --cuda-graph-max-bs-decode 128 \ |
| --mem-fraction-static 0.8 \ |
| --tokenizer-worker-num 8 \ |
| --host 0.0.0.0 \ |
| --port 30000 |
| ``` |
|
|
| # Model Quantization |
|
|
| The model's routed experts were quantized from NVFP4 to MXFP4 using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). |
|
|
|
|
| **Quantization script:** |
|
|
| ```bash |
| python quantize_quark.py \ |
| --model_dir MiniMaxAI/MiniMax-M3.1-Flash \ |
| --quant_scheme mxfp4 \ |
| --output_dir amd/MiniMax-M3.1-Flash-Quark-MXFP4 \ |
| --skip_evaluation --model_export hf_format --file2file_quantization \ |
| --exclude_layers \ |
| '*lm_head' \ |
| '*vision_tower*' \ |
| '*multi_modal_projector*' \ |
| '*patch_merge_mlp*' \ |
| '*self_attn*' \ |
| '*block_sparse_moe.gate' \ |
| 'language_model.model.layers.[0-2].mlp.*' \ |
| '*block_sparse_moe.shared_experts.*' \ |
| '*dspark*' \ |
| --keep_excluded_layers_as_original_model_state |
| ``` |
|
|
| # Evaluation |
| The model was evaluated on GSM8K using vLLM and EvalScope. |
|
|
| ### Accuracy |
|
|
| <table> |
| <tr> |
| <td><strong>Benchmark</strong></td> |
| <td><strong>MiniMax/MiniMax-M3.1-Flash</strong></td> |
| <td><strong>amd/MiniMax-M3.1-Flash-Quark-MXFP4 (this model)</strong></td> |
| <td><strong>Recovery</strong></td> |
| </tr> |
| <tr> |
| <td>GSM8K (EvalScope, 4-shot)</td> |
| <td>97.2%</td> |
| <td>96.9%</td> |
| <td>99.69%</td> |
| </tr> |
| </table> |
|
|
|
|
| ### Reproduction |
|
|
| The GSM8K results were obtained using EvalScope and vLLM. |
|
|
| #### Launching server |
| ```bash |
| VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=1 HIP_VISIBLE_DEVICES=0,1,2,3 \ |
| vllm serve amd/MiniMax-M3.1-Flash-Quark-MXFP4 \ |
| --tensor-parallel-size 4 \ |
| --host 0.0.0.0 \ |
| --port 8101 \ |
| --trust-remote-code \ |
| --gpu-memory-utilization 0.9 \ |
| --max-model-len 131072 \ |
| --max-num-seqs 80 \ |
| --max-num-batched-tokens 32768 \ |
| --kv-cache-dtype fp8 \ |
| --reasoning-parser minimax_m3 \ |
| --moe-backend aiter |
| ``` |
|
|
|
|
| #### Evaluating model in a new terminal |
| ```bash |
| evalscope eval \ |
| --model amd/MiniMax-M3.1-Flash-Quark-MXFP4 \ |
| --api-url http://localhost:8101/v1/chat/completions \ |
| --api-key EMPTY \ |
| --datasets gsm8k \ |
| --dataset-hub huggingface \ |
| --dataset-args '{"gsm8k":{"dataset_id":"openai/gsm8k"}}' \ |
| --dataset-dir hf_hub_cache \ |
| --eval-batch-size 32 \ |
| --generation-config '{"temperature":1,"top_p":1,"max_tokens":8192,"timeout":60000,"extra_body":{"chat_template_kwargs":{"thinking_mode":"enabled"}}}' \ |
| --work-dir logs/gsm8k |
| ``` |
|
|
| # Others |
|
|
| ## Experimental MXFP4 shared experts |
|
|
| MXFP4 [online](https://docs.vllm.ai/en/stable/features/quantization/online/) shared experts quantization and shared experts fusion can optionally be enabled: |
|
|
| ```bash |
| VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 vllm serve \ |
| amd/MiniMax-M3.1-Flash-Quark-MXFP4 \ |
| ... \ |
| --quantization-config.targets '{"*shared_expert*": "mxfp4"}' |
| ``` |
|
|
| but is experimental and not the recommended default setup. |
|
|
| # License |
| Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved. |
|
|
