| --- |
| base_model: MiniMaxAI/MiniMax-M3.1-Flash |
| base_model_relation: quantized |
| quantized_by: AtomicChat |
| pipeline_tag: image-text-to-text |
| library_name: gguf |
| license: other |
| license_name: minimax-community |
| license_link: LICENSE |
| tags: |
| - gguf |
| - llama.cpp |
| - minimax |
| - minimax-m3 |
| - moe |
| - quantized |
| - conversational |
| - atomic-chat |
| --- |
| |
| # How to Run MiniMax M3.1 Flash Locally |
| <p style="margin-top: 0; margin-bottom: 0;"> |
| <em>Built with MiniMax M3.1-Flash. Converted from MiniMax's original weights on release day.</em> |
| </p> |
| <div style="display: flex; gap: 8px; align-items: center; margin-top: 10px; margin-bottom: 10px;"> |
| <a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_minimax_m3_1_flash&utm_content=btn_atomic"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_atomic.png" width="162" alt="Atomic Chat"></a> |
| <a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_discord.png" width="119" alt="Discord"></a> |
| <a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_github.png" width="115" alt="GitHub"></a> |
| <a href="https://x.com/atomic_chat_hq"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_x.png" width="108" alt="X"></a> |
| </div> |
| <ul style="margin: 0 0 12px 0;"> |
| <li>You can now run MiniMax M3.1 Flash in <a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_minimax_m3_1_flash&utm_content=bullet_app">Atomic Chat</a>, locally on your own machine.</li> |
| </ul> |
| <hr style="margin: 0 0 16px 0;"> |
|
|
| > [!NOTE] |
| > **Atomic Dynamic files.** The routed experts carry the compression; attention, the shared expert and the |
| > embeddings stay high; an importance matrix computed on M3.1 Flash's own chat format guides the rounding. |
| > They replace the plain llama.cpp quants we published on release day: at each size the AD file stays |
| > closer to the original (see Quality). |
| > |
| > All of them load in stock llama.cpp and the apps built on it: standard `minimax-m3` architecture, no runtime patch. |
|
|
| ## Pick a file |
|
|
| | File | Size on disk | Memory for the model | Fits | |
| |---|---:|---:|---| |
| | `AD-IQ2_XXS` | 112.2 GB | 104.5 GiB | 128 GB machines: a 128 GB Mac with the GPU limit raised (context up to about 32k), a DGX Spark, a PC with 128 GB of RAM and a GPU | |
| | `AD-Q2_K` | 161.6 GB | 150.5 GiB | 192 GB machines | |
| | `AD-IQ3_S` | 208.9 GB | 194.6 GiB | 256 GB machines | |
| | `AD-Q4_K` | 257.2 GB | 239.5 GiB | 384 GB machines and up, or several GPUs | |
|
|
| Download one of them, for example: |
|
|
| ```bash |
| hf download AtomicChat/MiniMax-M3.1-Flash-GGUF --include "AD-IQ3_S/*" --local-dir MiniMax-M3.1-Flash |
| ``` |
|
|
| MiniMax M3.1 Flash has 428B parameters, 23B of them active per token, so it needs a lot of memory but runs |
| fast for its size. Add memory for the context: the attention cache takes about 0.14 MB per token, which is |
| 1.1 GB at 8k context and 4.5 GB at 32k. Memory is in GiB (1 GiB = 1.074 GB), which is how GPUs and Macs report |
| it; sizes on disk are in GB. |
|
|
| > [!NOTE] |
| > On a Mac, macOS lets the GPU use only part of the unified memory by default. Raise the limit before loading, |
| > for example `sudo sysctl iogpu.wired_limit_mb=245000` on a 256 GB Mac (resets on reboot). |
|
|
| ## Quality |
|
|
| Measured against Q8_0 with llama.cpp `f5027fb21307`; a dash means not measured yet, and the table fills in as |
| the runs finish. |
| |
| | File | Size | Mean KLD | Top-1 agreement | Chat checks | |
| |---|---:|---:|---:|---| |
| | `Q4_K_M` (release-day stock, replaced) | 258.0 GB | 0.9548 | 75.2% | — | |
| | `AD-Q4_K` | 257.2 GB | 0.7884 | 78.6% | 4/4 | |
| | `AD-IQ3_S` | 208.9 GB | 0.8916 | 75.9% | 4/4 | |
| | `Q3_K_M` (release-day stock, replaced) | 203.5 GB | 1.1442 | 70.9% | 4/4 | |
| | `AD-Q2_K` | 161.6 GB | 1.1346 | 71.1% | 4/4 | |
| | `Q2_K` (release-day stock, replaced) | 155.9 GB | 1.6387 | 62.0% | — | |
| | `AD-IQ2_XXS` | 112.2 GB | 1.8437 | 59.8% | 4/4 | |
|
|
| - **Mean KLD** is how far a file's next-token probabilities are from Q8_0's, on held-out text in M3.1 Flash's own chat format (half of our calibration build, which the importance matrix |
| did not see: agentic, code, reasoning, 30 languages), 4096-token context. Lower is better. **Top-1 agreement** is how often the file picks the same next token |
| as Q8_0. |
| - The stock rows are the plain llama.cpp quants we published on release day, kept here for comparison: |
| at each size the AD file of the same class is closer to the original. |
| - These levels are far from what other models show at the same sizes: even the 4-bit files pick the |
| reference's top token about three times in four. Compare the files with each other here, not with other |
| models' cards; we are looking into why (experts trained for FP4, sparse attention). |
| - No perplexity here: on plain text this chat model's perplexity is erratic and even comes out lower for the |
| smaller files, so it says nothing about their quality. |
| - **Chat checks**: a tool call, an answer in a language other than English, a JSON answer and arithmetic at low reasoning effort, |
| at MiniMax's sampling. |
| - The reference here is Q8_0, made from our BF16 copy of MiniMax's MXFP8/NVFP4 checkpoint (see below): the BF16 does |
| not fit the memory of the machine we measure on. Q8_0 is close to BF16, so the KLD of the other files reads |
| slightly low; numbers against BF16 will replace these. |
|
|
|
|
| ## Running it |
|
|
| ```bash |
| llama-server -m MiniMax-M3.1-Flash/AD-IQ3_S/MiniMax-M3.1-Flash-AD-IQ3_S-00001-of-00005.gguf -ngl 99 -c 32768 -fa on --jinja |
| ``` |
|
|
| or straight from the hub: `llama-server -hf AtomicChat/MiniMax-M3.1-Flash-GGUF:AD-IQ3_S -ngl 99 -c 32768 -fa on --jinja`. |
|
|
| The chat template is inside every file; `--jinja` applies it. The files are split into parts of up to 45 GB; |
| point llama.cpp at the first part and it loads the rest. |
|
|
| - **Sparse attention.** M3.1 Flash uses MiniMax Sparse Attention. llama.cpp runs it only with flash attention |
| on (`-fa on`) and without `--kv-unified`; otherwise it falls back to dense attention and warns about it. |
| - **Thinking.** The model decides on its own when to think. Set the depth with |
| `--chat-template-kwargs '{"reasoning_effort":"high"}'` (`low`, `medium`, `high`, `xhigh` or `max`). |
| - **Sampling.** MiniMax recommends `--temp 1.0 --top-p 0.95`. |
| - **Images.** `mmproj-MiniMax-M3.1-Flash-F16.gguf` (1.7 GB) is the vision encoder: add |
| `--mmproj mmproj-MiniMax-M3.1-Flash-F16.gguf`. It is converted from the original weights; we have not tested |
| images with it yet. |
| - **llama.cpp.** MiniMax-M3 support landed in July 2026; these files were made and checked with |
| `f5027fb21307` (23 October 2026). |
| - **Checked:** every file answered a chat turn through the built-in template before upload; measured quality is in the |
| table above. |
|
|
| ## How these files were made |
|
|
| MiniMax ships M3.1 Flash with its dense weights in MXFP8 and its routed experts in NVFP4 (trained for it). |
| llama.cpp's converter does not read this combination yet, so we first unpacked the weights to BF16: the MXFP8 |
| values are exact in BF16, and the NVFP4 values carry a per-tensor scale that rounds at about 0.2% per weight. |
| From that BF16 copy, upstream `convert_hf_to_gguf.py` built the GGUF and `llama-quantize` built the files above. |
|
|
| One difference from the original: M3.1 Flash uses sparse attention in all 60 blocks, while llama.cpp's |
| MiniMax-M3 code runs the first three blocks with dense attention, so their sparse-attention indexer weights are |
| not in these files. While the context fits the blocks the indexer would pick (16 blocks of 128 tokens plus the |
| local block, about 2,200 tokens), the result is the same; with a longer context those three blocks look at all |
| of it instead of their usual selection. |
|
|
| The `AD-*` files come from the same BF16 GGUF. Their importance matrix comes from `llama-imatrix` over about 2.4M tokens |
| (every other piece of a 4.75M-token build of agentic, code, reasoning, 30-language and long-document text |
| rendered through M3.1 Flash's own chat template), computed on a Q8_0 copy of the model because the BF16 does |
| not fit our GPU machine. Every one of the 128 experts in every block received tokens. In each file the |
| routed-expert blocks with the highest activation energy (the last six) get one step more precision and |
| the coldest early blocks one step less. |
| |
| The DSpark draft model that MiniMax ships for speculative decoding is not included. |
| |
| ## License |
| |
| These files are a derivative of MiniMax M3.1 Flash and come under the |
| [MiniMax Community License](LICENSE). It allows non-commercial use freely. Commercial use requires showing |
| "Built with MiniMax M3.1-Flash" and a one-time notice to api@minimax.io, and written authorization from MiniMax |
| above 20 million US dollars of yearly revenue; see the license for the full terms and the prohibited uses. |
| |
| Quantized by [AtomicChat](https://atomic.chat). |
| |
