Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf LyJonathan/MiniMax-M3.1-Flash-GGUF:# Run inference directly in the terminal:
llama cli -hf LyJonathan/MiniMax-M3.1-Flash-GGUF:Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf LyJonathan/MiniMax-M3.1-Flash-GGUF:# Run inference directly in the terminal:
./llama-cli -hf LyJonathan/MiniMax-M3.1-Flash-GGUF:Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf LyJonathan/MiniMax-M3.1-Flash-GGUF:# Run inference directly in the terminal:
./build/bin/llama-cli -hf LyJonathan/MiniMax-M3.1-Flash-GGUF:Use Docker
docker model run hf.co/LyJonathan/MiniMax-M3.1-Flash-GGUF:MiniMax-M3.1-Flash GGUF
Experimental. These quants and the runtime are a work in progress. Expect bugs, slow speeds and changes.
Use the Sparsa engine. Most of these files do not load in stock llama.cpp, LM Studio or Ollama. Download Sparsa from its releases page.
Quants
Each quant is a folder of 13 parts; point Sparsa at the folder. "bpw" = bits per weight. Quality is measured against the full-precision model (higher top-1 = closer; see below).
| Quant | Size | bpw | Top-1 | Loads in |
|---|---|---|---|---|
| IQ1_S | 94.8 GB | 1.78 | 44.3 % ⚠️ | Sparsa · llama.cpp (patched) |
| IQ1_M | 106.6 GB | 2.00 | 47.1 % | Sparsa · llama.cpp (patched) |
| IQ2_XXS | 124.5 GB | 2.34 | 52.0 % | Sparsa · llama.cpp (patched) |
| IQ2_KT | 123.8 GB | 2.32 | 56.3 % | Sparsa only |
| IQ2_XS | 133.1 GB | 2.50 | see ¹ | Sparsa · llama.cpp (patched) |
| IQ2_XS-D3 | 146.1 GB | 2.74 | 59.4 % | Sparsa · llama.cpp (patched) |
| IQ2_KL | 152.7 GB | 2.87 | 59.8 % | Sparsa only |
| IQ3_K | 191.2 GB | 3.59 | 69.8 % | Sparsa only |
| IQ4_KS | 233.6 GB | 4.38 | 74.4 % | Sparsa only |
| NVFP4 | 246.1 GB | 4.62 | 77.3 % | Sparsa only |
⚠️ IQ1_S is broken in practice: in our test it loops ("and and and…") instead of writing text. Use IQ1_M or bigger.
Which one? Pick the biggest that fits in your RAM + VRAM with ~10 GB to spare. Sparsa streams whatever doesn't fit from the SSD, which works but is slower.
Vision: mmproj-MiniMax-M3.1-Flash-F16.gguf (1.73 GB) adds image input for every quant.
Experimental, CPU-only for now.
The Hub's "Hardware compatibility" box doesn't recognise most of these formats, so it hides or mislabels several quants. The table above is the real list.
Quick start
git clone https://github.com/LyJonathan/Sparsa.git && cd Sparsa
# unzip the release for your OS into bin/, then:
python -m pip install -r requirements.txt
python sparsa.py download iq2-xs-d3 --dir D:/models
python sparsa.py download tokenizer --dir D:/models/MiniMax-M3.1-Flash-IQ2_XS-D3/tokenizer
python sparsa.py serve --model D:/models/MiniMax-M3.1-Flash-IQ2_XS-D3
This gives you OpenAI- and Anthropic-compatible APIs on http://127.0.0.1:8080. Add --access lan
to share it on your network with an API key. Full instructions are in the
Sparsa README.
Measured speed (IQ2_XS-D3, 2× RTX 5070 12 GB, Ryzen 9 5900X, 128 GB DDR4, one NVMe): about 4 tok/s writing, 13 tok/s reading a 400-token prompt, 44 tok/s on a 32K prompt. Speed depends heavily on your SSD.
Quality
Teacher-forced against an FP32 reference of the original weights, 6 short prompts (679 positions), run in Sparsa. Top-1 = how often the quant's top next-token matches the full model's. KL64 = divergence over the reference's top 64 tokens (lower is better).
| IQ1_S | IQ1_M | IQ2_KT | IQ2_XXS | IQ2_XS-D3 | IQ2_KL | IQ3_K | IQ4_KS | NVFP4 | |
|---|---|---|---|---|---|---|---|---|---|
| Top-1 | 44.3 % | 47.1 % | 56.3 % | 52.0 % | 59.4 % | 59.8 % | 69.8 % | 74.4 % | 77.3 % |
| KL64 | 1.926 | 1.673 | 1.292 | 1.804 | 1.211 | 1.184 | 0.847 | 0.760 | 0.670 |
¹ IQ2_XS was measured in patched llama.cpp on a harder 8-prompt set: top-1 46.8 %, KL 1.92 (IQ2_XS-D3 on the same set: 53.8 %, 1.58). Not comparable with the table above.
One hard code prompt pulls every mean down; on the other five, NVFP4 agrees 74–95 % of the time. The 1–2-bit quants are noticeably degraded; the experts were trained as 4-bit (NVFP4).
How they're made
- Routed experts are quantized at the listed type, with an imatrix from 23,738 calibration tokens.
The small quants put
ffn_down_expsone type higher. - Attention, indexer, dense layers and shared experts stay at Q8_0 (Q6_K in IQ1_S/IQ1_M); output is Q6_K.
- NVFP4 is a lossless copy of the checkpoint's 4-bit experts.
- llama.cpp (patched) means it needs the small MiniMax-M3 patch that ships with Sparsa. Stock llama.cpp runs layers 0–2 with dense attention.
- Checksums are in
SHA256SUMS(sha256sum -c SHA256SUMS).
License
MiniMax Community License (LICENSE). Commercial use requires displaying
"Built with MiniMax M3.1-Flash" and a notice to api@minimax.io (separate authorization above $20M
yearly revenue). The license's prohibited uses apply.
Credits
MiniMax (model) · llama.cpp/ggml (format, quantization) · ik_llama.cpp by Iwan Kawrakow (IQ_K/KT types) · Strata by Niko1221 (engine base) · small-quant recipes inspired by Unsloth's dynamic quants and ubergarm's ik_llama.cpp recipes (no code copied).
- Downloads last month
- 12,391
1-bit
2-bit
4-bit
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf LyJonathan/MiniMax-M3.1-Flash-GGUF:# Run inference directly in the terminal: llama cli -hf LyJonathan/MiniMax-M3.1-Flash-GGUF: