metadata
base_model:
- MiniMaxAI/MiniMax-M3.1-Flash
license: other
license_name: minimax-community
license_link: LICENSE
Model Overview
- Model Architecture: MiniMaxM3SparseForConditionalGeneration
- Input: Text, Image, Video
- Output: Text
- Supported Hardware Microarchitecture: AMD MI350/MI355
- ROCm: 7.2.3
- PyTorch: 2.13.0
- Transformers: 5.19.0
- Operating System(s): Linux
- Inference Engine: vLLM, SGLang
- Model Optimizer: AMD-Quark
- Routed-expert weight quantization: OCP MXFP4, Static
- Routed-expert activation quantization: OCP MXFP4, Dynamic
- All other layers: unchanged precision compared to base MiniMaxAI/MiniMax-M3.1-Flash
Model serving
Please refer to vLLM recipe and SGLang cookbook.
Example usage:
vLLM
VLLM_ROCM_USE_AITER=1 \
vllm serve amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
--tensor-parallel-size 4 \
--reasoning-parser minimax_m3 \
--block-size 128 \
--speculative-config '{"method":"dspark","model":"amd/MiniMax-M3.1-Flash-Quark-MXFP4/dspark","num_speculative_tokens":7,"attention_backend":"FLASHINFER_TS","kv_cache_dtype":"fp8","draft_sample_method":"probabilistic","rejection_sample_method":"block"}' \
--disable-hybrid-kv-cache-manager"
SGLang
SGLANG_USE_AITER=1 \
sglang serve \
--trust-remote-code \
--model-path amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
--reasoning-parser minimax-m3 \
--tool-call-parser minimax-m3 \
--tp 4 \
--speculative-algorithm DSPARK \
--speculative-use-block-verification \
--cuda-graph-backend-prefill breakable \
--fp8-gemm-backend aiter \
--chunked-prefill-size 32768 \
--max-running-requests 128 \
--cuda-graph-max-bs-decode 128 \
--mem-fraction-static 0.8 \
--tokenizer-worker-num 8 \
--host 0.0.0.0 \
--port 30000
Model Quantization
The model's routed experts were quantized from NVFP4 to MXFP4 using AMD-Quark.
Quantization script:
python quantize_quark.py \
--model_dir MiniMaxAI/MiniMax-M3.1-Flash \
--quant_scheme mxfp4 \
--output_dir amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
--skip_evaluation --model_export hf_format --file2file_quantization \
--exclude_layers \
'*lm_head' \
'*vision_tower*' \
'*multi_modal_projector*' \
'*patch_merge_mlp*' \
'*self_attn*' \
'*block_sparse_moe.gate' \
'language_model.model.layers.[0-2].mlp.*' \
'*block_sparse_moe.shared_experts.*' \
'*dspark*' \
--keep_excluded_layers_as_original_model_state
Evaluation
The model was evaluated on GSM8K using vLLM and EvalScope.
Accuracy
| Benchmark | MiniMax/MiniMax-M3.1-Flash | amd/MiniMax-M3.1-Flash-Quark-MXFP4 (this model) | Recovery |
| GSM8K (EvalScope, 4-shot) | 97.2% | 96.9% | 99.69% |
Reproduction
The GSM8K results were obtained using EvalScope and vLLM.
Launching server
VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=1 HIP_VISIBLE_DEVICES=0,1,2,3 \
vllm serve amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
--tensor-parallel-size 4 \
--host 0.0.0.0 \
--port 8101 \
--trust-remote-code \
--gpu-memory-utilization 0.9 \
--max-model-len 131072 \
--max-num-seqs 80 \
--max-num-batched-tokens 32768 \
--kv-cache-dtype fp8 \
--reasoning-parser minimax_m3 \
--moe-backend aiter
Evaluating model in a new terminal
evalscope eval \
--model amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
--api-url http://localhost:8101/v1/chat/completions \
--api-key EMPTY \
--datasets gsm8k \
--dataset-hub huggingface \
--dataset-args '{"gsm8k":{"dataset_id":"openai/gsm8k"}}' \
--dataset-dir hf_hub_cache \
--eval-batch-size 32 \
--generation-config '{"temperature":1,"top_p":1,"max_tokens":8192,"timeout":60000,"extra_body":{"chat_template_kwargs":{"thinking_mode":"enabled"}}}' \
--work-dir logs/gsm8k
Others
Experimental MXFP4 shared experts
MXFP4 online shared experts quantization and shared experts fusion can optionally be enabled:
VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 vllm serve \
amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
... \
--quantization-config.targets '{"*shared_expert*": "mxfp4"}'
but is experimental and not the recommended default setup.
License
Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.