LyJonathan/MiniMax-M3.1-Flash-GGUF

MiniMax-M3.1-Flash GGUF

Experimental. These quants and the runtime are a work in progress. Expect bugs, slow speeds and changes.

Use the Sparsa engine. Most of these files do not load in stock llama.cpp, LM Studio or Ollama. Download Sparsa from its releases page.

Quants

Each quant is a folder of 13 parts; point Sparsa at the folder. "bpw" = bits per weight. Quality is measured against the full-precision model (higher top-1 = closer; see below).

Quant	Size	bpw	Top-1	Loads in
IQ1_S	94.8 GB	1.78	44.3 % ⚠️	Sparsa · llama.cpp (patched)
IQ1_M	106.6 GB	2.00	47.1 %	Sparsa · llama.cpp (patched)
IQ2_XXS	124.5 GB	2.34	52.0 %	Sparsa · llama.cpp (patched)
IQ2_KT	123.8 GB	2.32	56.3 %	Sparsa only
IQ2_XS	133.1 GB	2.50	see ¹	Sparsa · llama.cpp (patched)
IQ2_XS-D3	146.1 GB	2.74	59.4 %	Sparsa · llama.cpp (patched)
IQ2_KL	152.7 GB	2.87	59.8 %	Sparsa only
IQ3_K	191.2 GB	3.59	69.8 %	Sparsa only
IQ4_KS	233.6 GB	4.38	74.4 %	Sparsa only
NVFP4	246.1 GB	4.62	77.3 %	Sparsa only

⚠️ IQ1_S is broken in practice: in our test it loops ("and and and…") instead of writing text. Use IQ1_M or bigger.

Which one? Pick the biggest that fits in your RAM + VRAM with ~10 GB to spare. Sparsa streams whatever doesn't fit from the SSD, which works but is slower.

Vision: mmproj-MiniMax-M3.1-Flash-F16.gguf (1.73 GB) adds image input for every quant. Experimental, CPU-only for now.

The Hub's "Hardware compatibility" box doesn't recognise most of these formats, so it hides or mislabels several quants. The table above is the real list.

Quick start

git clone https://github.com/LyJonathan/Sparsa.git && cd Sparsa
# unzip the release for your OS into bin/, then:
python -m pip install -r requirements.txt
python sparsa.py download iq2-xs-d3 --dir D:/models
python sparsa.py download tokenizer --dir D:/models/MiniMax-M3.1-Flash-IQ2_XS-D3/tokenizer
python sparsa.py serve --model D:/models/MiniMax-M3.1-Flash-IQ2_XS-D3

This gives you OpenAI- and Anthropic-compatible APIs on http://127.0.0.1:8080. Add --access lan to share it on your network with an API key. Full instructions are in the Sparsa README.

Measured speed (IQ2_XS-D3, 2× RTX 5070 12 GB, Ryzen 9 5900X, 128 GB DDR4, one NVMe): about 4 tok/s writing, 13 tok/s reading a 400-token prompt, 44 tok/s on a 32K prompt. Speed depends heavily on your SSD.

Quality

Teacher-forced against an FP32 reference of the original weights, 6 short prompts (679 positions), run in Sparsa. Top-1 = how often the quant's top next-token matches the full model's. KL64 = divergence over the reference's top 64 tokens (lower is better).

	IQ1_S	IQ1_M	IQ2_KT	IQ2_XXS	IQ2_XS-D3	IQ2_KL	IQ3_K	IQ4_KS	NVFP4
Top-1	44.3 %	47.1 %	56.3 %	52.0 %	59.4 %	59.8 %	69.8 %	74.4 %	77.3 %
KL64	1.926	1.673	1.292	1.804	1.211	1.184	0.847	0.760	0.670

¹ IQ2_XS was measured in patched llama.cpp on a harder 8-prompt set: top-1 46.8 %, KL 1.92 (IQ2_XS-D3 on the same set: 53.8 %, 1.58). Not comparable with the table above.

One hard code prompt pulls every mean down; on the other five, NVFP4 agrees 74–95 % of the time. The 1–2-bit quants are noticeably degraded; the experts were trained as 4-bit (NVFP4).

How they're made

• Routed experts are quantized at the listed type, with an imatrix from 23,738 calibration tokens. The small quants put ffn_down_exps one type higher.
• Attention, indexer, dense layers and shared experts stay at Q8_0 (Q6_K in IQ1_S/IQ1_M); output is Q6_K.
• NVFP4 is a lossless copy of the checkpoint's 4-bit experts.
• llama.cpp (patched) means it needs the small MiniMax-M3 patch that ships with Sparsa. Stock llama.cpp runs layers 0–2 with dense attention.
• Checksums are in SHA256SUMS (sha256sum -c SHA256SUMS).

License

MiniMax Community License (LICENSE). Commercial use requires displaying "Built with MiniMax M3.1-Flash" and a notice to api@minimax.io (separate authorization above $20M yearly revenue). The license's prohibited uses apply.

Credits

MiniMax (model) · llama.cpp/ggml (format, quantization) · ik_llama.cpp by Iwan Kawrakow (IQ_K/KT types) · Strata by Niko1221 (engine base) · small-quant recipes inspired by Unsloth's dynamic quants and ubergarm's ik_llama.cpp recipes (no code copied).