--- base_model: MiniMaxAI/MiniMax-M3.1-Flash base_model_relation: quantized quantized_by: AtomicChat pipeline_tag: image-text-to-text library_name: gguf license: other license_name: minimax-community license_link: LICENSE tags: - gguf - llama.cpp - minimax - minimax-m3 - moe - quantized - conversational - atomic-chat --- # How to Run MiniMax M3.1 Flash Locally

Built with MiniMax M3.1-Flash. Converted from MiniMax's original weights on release day.

Atomic Chat Discord GitHub X
> [!NOTE] > **Atomic Dynamic files.** The routed experts carry the compression; attention, the shared expert and the > embeddings stay high; an importance matrix computed on M3.1 Flash's own chat format guides the rounding. > They replace the plain llama.cpp quants we published on release day: at each size the AD file stays > closer to the original (see Quality). > > All of them load in stock llama.cpp and the apps built on it: standard `minimax-m3` architecture, no runtime patch. ## Pick a file | File | Size on disk | Memory for the model | Fits | |---|---:|---:|---| | `AD-IQ2_XXS` | 112.2 GB | 104.5 GiB | 128 GB machines: a 128 GB Mac with the GPU limit raised (context up to about 32k), a DGX Spark, a PC with 128 GB of RAM and a GPU | | `AD-Q2_K` | 161.6 GB | 150.5 GiB | 192 GB machines | | `AD-IQ3_S` | 208.9 GB | 194.6 GiB | 256 GB machines | | `AD-Q4_K` | 257.2 GB | 239.5 GiB | 384 GB machines and up, or several GPUs | Download one of them, for example: ```bash hf download AtomicChat/MiniMax-M3.1-Flash-GGUF --include "AD-IQ3_S/*" --local-dir MiniMax-M3.1-Flash ``` MiniMax M3.1 Flash has 428B parameters, 23B of them active per token, so it needs a lot of memory but runs fast for its size. Add memory for the context: the attention cache takes about 0.14 MB per token, which is 1.1 GB at 8k context and 4.5 GB at 32k. Memory is in GiB (1 GiB = 1.074 GB), which is how GPUs and Macs report it; sizes on disk are in GB. > [!NOTE] > On a Mac, macOS lets the GPU use only part of the unified memory by default. Raise the limit before loading, > for example `sudo sysctl iogpu.wired_limit_mb=245000` on a 256 GB Mac (resets on reboot). ## Quality Measured against Q8_0 with llama.cpp `f5027fb21307`; a dash means not measured yet, and the table fills in as the runs finish. | File | Size | Mean KLD | Top-1 agreement | Chat checks | |---|---:|---:|---:|---| | `Q4_K_M` (release-day stock, replaced) | 258.0 GB | 0.9548 | 75.2% | — | | `AD-Q4_K` | 257.2 GB | 0.7884 | 78.6% | 4/4 | | `AD-IQ3_S` | 208.9 GB | 0.8916 | 75.9% | 4/4 | | `Q3_K_M` (release-day stock, replaced) | 203.5 GB | 1.1442 | 70.9% | 4/4 | | `AD-Q2_K` | 161.6 GB | 1.1346 | 71.1% | 4/4 | | `Q2_K` (release-day stock, replaced) | 155.9 GB | 1.6387 | 62.0% | — | | `AD-IQ2_XXS` | 112.2 GB | 1.8437 | 59.8% | 4/4 | - **Mean KLD** is how far a file's next-token probabilities are from Q8_0's, on held-out text in M3.1 Flash's own chat format (half of our calibration build, which the importance matrix did not see: agentic, code, reasoning, 30 languages), 4096-token context. Lower is better. **Top-1 agreement** is how often the file picks the same next token as Q8_0. - The stock rows are the plain llama.cpp quants we published on release day, kept here for comparison: at each size the AD file of the same class is closer to the original. - These levels are far from what other models show at the same sizes: even the 4-bit files pick the reference's top token about three times in four. Compare the files with each other here, not with other models' cards; we are looking into why (experts trained for FP4, sparse attention). - No perplexity here: on plain text this chat model's perplexity is erratic and even comes out lower for the smaller files, so it says nothing about their quality. - **Chat checks**: a tool call, an answer in a language other than English, a JSON answer and arithmetic at low reasoning effort, at MiniMax's sampling. - The reference here is Q8_0, made from our BF16 copy of MiniMax's MXFP8/NVFP4 checkpoint (see below): the BF16 does not fit the memory of the machine we measure on. Q8_0 is close to BF16, so the KLD of the other files reads slightly low; numbers against BF16 will replace these. ## Running it ```bash llama-server -m MiniMax-M3.1-Flash/AD-IQ3_S/MiniMax-M3.1-Flash-AD-IQ3_S-00001-of-00005.gguf -ngl 99 -c 32768 -fa on --jinja ``` or straight from the hub: `llama-server -hf AtomicChat/MiniMax-M3.1-Flash-GGUF:AD-IQ3_S -ngl 99 -c 32768 -fa on --jinja`. The chat template is inside every file; `--jinja` applies it. The files are split into parts of up to 45 GB; point llama.cpp at the first part and it loads the rest. - **Sparse attention.** M3.1 Flash uses MiniMax Sparse Attention. llama.cpp runs it only with flash attention on (`-fa on`) and without `--kv-unified`; otherwise it falls back to dense attention and warns about it. - **Thinking.** The model decides on its own when to think. Set the depth with `--chat-template-kwargs '{"reasoning_effort":"high"}'` (`low`, `medium`, `high`, `xhigh` or `max`). - **Sampling.** MiniMax recommends `--temp 1.0 --top-p 0.95`. - **Images.** `mmproj-MiniMax-M3.1-Flash-F16.gguf` (1.7 GB) is the vision encoder: add `--mmproj mmproj-MiniMax-M3.1-Flash-F16.gguf`. It is converted from the original weights; we have not tested images with it yet. - **llama.cpp.** MiniMax-M3 support landed in July 2026; these files were made and checked with `f5027fb21307` (23 October 2026). - **Checked:** every file answered a chat turn through the built-in template before upload; measured quality is in the table above. ## How these files were made MiniMax ships M3.1 Flash with its dense weights in MXFP8 and its routed experts in NVFP4 (trained for it). llama.cpp's converter does not read this combination yet, so we first unpacked the weights to BF16: the MXFP8 values are exact in BF16, and the NVFP4 values carry a per-tensor scale that rounds at about 0.2% per weight. From that BF16 copy, upstream `convert_hf_to_gguf.py` built the GGUF and `llama-quantize` built the files above. One difference from the original: M3.1 Flash uses sparse attention in all 60 blocks, while llama.cpp's MiniMax-M3 code runs the first three blocks with dense attention, so their sparse-attention indexer weights are not in these files. While the context fits the blocks the indexer would pick (16 blocks of 128 tokens plus the local block, about 2,200 tokens), the result is the same; with a longer context those three blocks look at all of it instead of their usual selection. The `AD-*` files come from the same BF16 GGUF. Their importance matrix comes from `llama-imatrix` over about 2.4M tokens (every other piece of a 4.75M-token build of agentic, code, reasoning, 30-language and long-document text rendered through M3.1 Flash's own chat template), computed on a Q8_0 copy of the model because the BF16 does not fit our GPU machine. Every one of the 128 experts in every block received tokens. In each file the routed-expert blocks with the highest activation energy (the last six) get one step more precision and the coldest early blocks one step less. The DSpark draft model that MiniMax ships for speculative decoding is not included. ## License These files are a derivative of MiniMax M3.1 Flash and come under the [MiniMax Community License](LICENSE). It allows non-commercial use freely. Commercial use requires showing "Built with MiniMax M3.1-Flash" and a one-time notice to api@minimax.io, and written authorization from MiniMax above 20 million US dollars of yearly revenue; see the license for the full terms and the prohibited uses. Quantized by [AtomicChat](https://atomic.chat).