| --- |
| language: |
| - en |
| - zh |
| library_name: mlx |
| license: other |
| license_name: minimax-community |
| license_link: LICENSE |
| pipeline_tag: image-text-to-text |
| base_model: MiniMaxAI/MiniMax-M3.1-Flash |
| tags: |
| - mlx |
| - jang |
| - janght |
| - quantized |
| - apple-silicon |
| - minimax |
| - minimax_m3 |
| - moe |
| - reasoning |
| - thinking |
| - agent |
| - tool-use |
| - vision |
| - speculative-decoding |
| - dspark |
| - 128gb |
| --- |
| |
| <p align="center"> |
| <a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a> |
| </p> |
|
|
| > **Runtime:** Osaurus **0.25.30 or newer** (`osaurus.json`: `required_osaurus_version` 0.25.30, `model_version` 1). |
| |
| # OsaurusAI/MiniMax-M3.1-Flash-JANGHT1.9 |
| |
| **MiniMax-M3.1-Flash for 128 GB Macs.** The bundle is 95.5 GiB, all of it held in memory: the model 93.3 GiB |
| (including its 1.6 GiB vision tower) and its DSpark speculative drafter 2.1 GiB. Image input, tool calls and adaptive |
| thinking with five reasoning efforts are supported. |
| |
| A JANGHT bundle of [MiniMaxAI/MiniMax-M3.1-Flash](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash). The source is |
| a 60-layer MoE (hidden 6144, 64 query / 4 KV heads, partial RoPE) with MiniMax sparse attention (a lightning indexer |
| picks the top 16 blocks of 128 keys for each query), 128 routed experts per layer (top-4 plus a shared expert) on layers |
| 3–59 and dense MLPs on layers 0–2, a 32-layer vision tower, and a 5-layer DSpark drafter. About 427B parameters |
| outside the drafter; the routed experts are 413B (97 %) of them. |
| |
| - **Routed experts** (84.07 GiB, 1.748 bits/weight): JANGT trellis codes, Hadamard-128 rotation with per-layer column |
| signs, error feedback from each expert's own activation Hessian, per-row gains refit, exact outlier columns. Widths |
| were allocated per layer and projection from each layer's measured effect on the assistant-token KL: gate/up 1 bit in |
| 16 layers, 1.5 in 26, 2 in 15; down 1 bit in 2 layers, 1.5 in 12, 2 in 10, 2.5 in 33. |
| - **Attention (q / k / v / o), shared experts and the dense MLPs of layers 0–2**: affine 4-bit (group 64), GPTQ on each |
| layer's captured input statistics. |
| - **Sparse-attention indexer** (index q / k projections): the release's MXFP8, repacked without requantization. |
| - **Router gates and score-correction biases**: F32 as released; norms BF16. |
| - **Token embeddings and `lm_head`**: affine 6-bit (group 64); the head with GPTQ on its input statistics. |
| - **Vision tower, patch merge and projector**: the release's BF16 / F32 tensors byte for byte (523 tensors). |
| - **DSpark drafter** (`dspark/`, 5 layers, 7-token blocks): as released (MXFP8 layers, BF16 fc and Markov head). |
| |
| Bits/weight 1.878 = model bytes × 8 / source parameters (BF16-equivalent), with the vision tower included in both and |
| the DSpark drafter excluded from both (the name rounds it to 1.9). |
| |
| ## Quality vs the source model |
| |
| Teacher-forced on held-out conversations that were never used for calibration (290 conversations, 456,954 scored |
| positions), the bundle exactly as it ships against the source model at its release precision. KL is over the source's |
| top-128 tokens plus one tail bucket (on assistant tokens the source's tail mass is 0.44 %). |
| |
| ### Assistant tokens (what the model writes) |
| | held-out domain | positions | mean KL ↓ | median KL ↓ | p99 KL ↓ | top-1 ↑ | top-5 ↑ | |
| |---|---|---|---|---|---|---| |
| | agentic (tool calls, multi-turn) | 34,066 | 0.128 | 0.022 | 1.58 | 88.8 % | 99.3 % | |
| | coding | 48,223 | 0.140 | 0.0008 | 2.41 | 91.3 % | 99.1 % | |
| | terminal | 29,650 | 0.048 | 0.0022 | 0.55 | 93.6 % | 99.9 % | |
| | reasoning | 29,068 | 0.091 | 0.0014 | 1.31 | 92.6 % | 99.7 % | |
| | long context (documents to 16k tokens) | 782 | 0.143 | 0.0072 | 2.17 | 89.5 % | 99.9 % | |
| | science | 16,236 | 0.251 | 0.043 | 2.76 | 84.5 % | 98.3 % | |
| | story (creative writing) | 36,199 | 0.611 | 0.216 | 5.62 | 71.9 % | 93.0 % | |
| | multilingual | 18,393 | 0.732 | 0.174 | 6.62 | 71.4 % | 91.5 % | |
| | **all** | **212,617** | **0.258** | **0.017** | **3.61** | **85.9 %** | **97.6 %** | |
| |
| Every scored position (system prompts, tool schemas, tool results, user text and raw documents included): mean KL |
| 1.236, median 0.085, top-1 70.1 %, top-5 85.4 %. That is text the model reads, not writes, and M3.1 itself is unsure |
| there (the source's mean NLL on the held-out text is 5.44 nats per token), so small differences flip many of those |
| positions; the KL there is also a looser bound (tail mass 12.7 %). The assistant tokens are the quality measure. |
| |
| - **The flips sit where the source itself is unsure.** Over all positions: where the source puts ≥ 0.99 on its top |
| token (26 % of positions) the bundle agrees 97.6 % of the time; at 0.9–0.99, 84.6 %; at 0.7–0.9, 72.6 %; at 0.5–0.7, |
| 58.2 %; below 0.5, 45–49 %. |
| - **Agentic, terminal, coding and reasoning hold up best.** Story and multilingual writing move the most (top-1 about |
| 71–72 %). Multilingual and science text also carried most of the trunk's error in the build's format tests. |
| - **The 4-bit trunk was chosen by measurement.** On a held-out subset (38 conversations), the 4-bit attention, shared |
| experts and dense MLPs together cost an assistant KL of 0.015 and the 6-bit head 0.0003; the same bytes were worth |
| more in the routed experts. |
| - Cybersecurity text was not in the calibration or held-out sets. |
| |
| ### Images |
| - **12 / 12 graded image prompts correct**: invoice OCR, shape counting, a bar chart, a table, a bug in a code |
| screenshot, a terminal error, layout and colour, arithmetic, Chinese OCR, a two-image comparison, a 1543 × 1999 |
| document page and a 112 px image (sizes not multiples of 28). |
| - Teacher-forced on the bundle's own answers with the same image features, against the source (1,764 answer tokens): |
| mean KL 0.073, median 0.0004, top-1 93.8 %, top-5 99.7 %. Think / skip choice 12 / 12 the same as the source. |
| - The routed experts were calibrated with image tokens too (452 image conversations at half weight in layers 3–31). |
| Without them an earlier build answered "Black" for a solid green image where the source answers "Green". |
| - One misread outside the graded fields: the invoice's shipping line $347.50 is read as $337.50. |
| |
| ### Reasoning (adaptive) |
| M3.1 decides for itself, turn by turn, whether to think: the first token of every reply is `<mm:think>` (think) or |
| `</mm:think>` (answer directly). At 327 held-out turn starts the bundle makes the same choice as the source **97.6 %** |
| of the time (the source thinks on 89.6 % of them, the bundle on 89.0 %; mean |ΔP(think)| 0.025). |
| |
| | domain | turn starts | same choice as the source | source thinks | bundle thinks | |
| |---|---|---|---|---| |
| | agentic | 86 | 96.5 % | 62.8 % | 61.6 % | |
| | coding | 49 | 91.8 % | 95.9 % | 95.9 % | |
| | terminal | 45 | 97.8 % | 100 % | 97.8 % | |
| | reasoning | 45 | 100 % | 100 % | 100 % | |
| | multilingual | 40 | 100 % | 100 % | 100 % | |
| | science | 33 | 100 % | 100 % | 100 % | |
| | story | 25 | 100 % | 100 % | 100 % | |
| | long context | 4 | 100 % | 100 % | 100 % | |
| |
| By reasoning effort: no effort tag 97.4 % (194 turns), `low` 100 % (24), `medium` 96.2 % (26), `high` 97.1 % (34), |
| `xhigh` 100 % (11), `max` 97.4 % (38). |
| |
| Raw results: `evaluation/kl_vs_source_heldout.json`, `evaluation/reasoning_decision.json`, |
| `evaluation/vision_graded.json`. |
|
|
| ## DSpark speculative decoding |
| The bundle ships the release's DSpark drafter unchanged (5 layers reading target layers 3, 17, 31, 45 and 59; 7-token |
| blocks; 2.1 GiB). Verification is exact: greedy output equals plain decoding, and sampled output follows the same |
| distribution as plain sampling (the draft only proposes; the target's own sample decides every token). Greedy output |
| with and without the drafter was token-identical on this bundle on coding, agentic, reasoning and terminal prompts. |
| Osaurus picks the draft width each round (Adaptive). |
|
|
| ## Speed and memory (Osaurus engine) |
| Osaurus serving path (the app's model loading, cache and batch engine), one request, prefill step 2048, document |
| prompts, cold cache. Medians of 6 probes per row over 2 interleaved rounds. |
|
|
| | | M5 Max 128 GB | M5 Ultra | |
| |---|---|---| |
| | Prefill, 4.5k-token prompt | 370-520 tok/s | 1,051 tok/s | |
| | Prefill, 35.7k-token prompt | 380-415 tok/s | 1,134 tok/s | |
| | Decode after a 4.5k prompt | 26-29 tok/s | 38.3 tok/s | |
| | Decode after a 35.7k prompt | 24-26 tok/s | 35.9 tok/s | |
| | Peak memory, 35.7k context | 99.6 GiB (102.5 with DSpark) | 99.6 GiB (102.2 with DSpark) | |
|
|
| The M5 Max was measured with the lid closed. The high end of each range is the first minutes; it settles to the low end |
| as the chip heats. At the same GPU clock an M5 Ultra prefills about 2x faster than an M5 Max. |
|
|
| DSpark is on by default and is exact. On code and other predictable output it speeds up decoding: 1.24-1.29x for |
| rewrite, list and simple-code prompts on an M5 Ultra, 1.06x on average over 15 chat prompts. On document summaries it |
| saves nothing: 0.97x on the M5 Ultra, about 0.88x on a heat-soaked M5 Max. Turn it off in the model picker |
| (Speculative Decoding → Off (AR)) for long-document work. |
|
|
| A follow-up on a cached conversation only prefills the new tokens. In the app, a follow-up question on a 35.7k-token |
| document prefilled 188 new tokens and restored the other 36,875 from the cache. After the app unloaded the idle model |
| and loaded it again, the next question still restored 37,059 tokens and prefilled 86. |
|
|
| ## Known limitations |
| - Very small images: for a 112 px solid red square (16 image tokens) with adaptive thinking, the bundle answered |
| "Purple". The same question with thinking Off and the graded image set (which includes a 112 px image) are answered |
| correctly. |
| - Nested tool arguments: asked to "run make test with CI=1 and FAST=0 and a timeout of 120 seconds" through a tool whose |
| schema has an optional `env` object and an integer `timeout_s`, the bundle wrote |
| `CI=1 FAST=0 timeout 120 make test` as the command and set `timeout_s` 130. The call is well-formed and valid against |
| the schema, but the optional `env` object was not used; agent loops that rely on nested objects should validate the |
| argument structure. |
|
|
| ## What's in the bundle |
| - **Vision**: the vision tower, patch merge and projector in-bundle (`config.json` → `vision_config`), with the release's |
| image and video processors (`preprocessor_config.json`, `image_processor.py`, `processing_minimax.py`, |
| `video_processor.py`). |
| - **DSpark drafter**: `dspark/` (`model-dspark-00001.safetensors` + `config.json`); Osaurus' speculative-decoding switch |
| turns it on or off. |
| - **Thinking + agentic** (`chat_template.jinja`): |
| - Adaptive by default: with no thinking setting the template renders the release's adaptive instructions and adds no |
| prefix, so the model decides each turn. |
| - `thinking_mode` chat-template kwarg: `"enabled"` (always think; the reply starts after `<mm:think>`) or |
| `"disabled"` (no thinking; the reply starts after `</mm:think>`), the same switch and wording as MiniMax-M3's |
| official template. Unset, the template renders exactly as the release's. Osaurus' thinking selector maps Default / |
| On / Off to unset / `enabled` / `disabled`. |
| - Reasoning effort via the `reasoning_effort` (or `effort`) kwarg: `low` / `medium` / `high` / `xhigh` / `max`; no |
| effort tag when unset. |
| - Past reasoning is rendered on every earlier assistant turn, so send each turn's `reasoning_content` back with the |
| history. |
| - Tool calls in the release's native XML format: one `<tool_call>` block holding `<invoke name="…">` elements with |
| one element per argument (nested objects and arrays expand into elements), every tag prefixed by the |
| `]<]minimax[>[` token; tool results go back in the `tool` role. |
| - Roles: a leading `root` message is the high-priority system block; `system` / `developer` messages map to the |
| lower-priority developer block. Without a `root` message the release's default system text is used. |
| - **Self-describing**: `config.json` carries the `jangt` expert-format block (trellis state, Hadamard rotation, |
| outlier columns per projection) and a per-module `quantization` map; `jang_config.json` carries the format, bits / |
| weight, source parameters, per-layer expert widths, trunk formats, embedding / head bits, sampling defaults and the |
| drafter note. `jang_config.format` reads `jangtq2` for loader compatibility; every routed expert is `mode: jangt`. |
| - `model.safetensors.index.json` covers the 23 model shards (3,416 tensors). |
| |
| ## Serving contract |
| - Sampling: the release defaults, `temperature=1.0, top_p=0.95` |
| - EOS: `[200020]` (`[e~[`) · BOS: 200019 · context: 1,048,576 |
| - Reasoning: adaptive by default; `thinking_mode` = `enabled` / `disabled`; `reasoning_effort` = `low` / `medium` / |
| `high` / `xhigh` / `max` |
|
|
| ## Build details |
| - Source: `MiniMaxAI/MiniMax-M3.1-Flash` (revision `262ae68`) |
| - Calibration: 1.86M tokens of multi-turn agentic tool use, terminal sessions, coding in 11 languages, reasoning, |
| long-context retrieval (documents to 16k tokens), multilingual text (Aya and Wikipedia in 8 languages), |
| encyclopedic prose, science and creative writing, rendered through the release template with think and skip turns |
| as the model takes them and a reasoning-effort tag on about a third of the rows; the source model's own adaptive |
| traces (424 agentic, terminal, coding, reasoning, science and multilingual prompts, the held-out ones kept for |
| evaluation); and 452 image conversations |
| (photos, charts, document pages, synthetic renders; 279k tokens) in the expert statistics of layers 3–31 at half |
| weight. Held-out sets disjoint (no shared rows or prompts) |
| - Experts: JANGT trellis (1 / 1.5 / 2 / 2.5 bits per weight per layer and projection), Hadamard-128 rotation, |
| Hessian-weighted error feedback, per-row gains refit, exact outlier columns; widths from measured per-layer |
| sensitivity (one held-out run per MoE layer, assistant-token KL), allocated greedily at the byte budget |
| - Everything else: 4-bit GPTQ trunk, MXFP8 indexer as released, 6-bit embeddings and `lm_head`, each chosen by its |
| held-out KL against what its bytes buy in the experts; vision tower and DSpark drafter as released |
| - License: MiniMax Community License (`LICENSE`), as the source |
|
|
| Quantized and validated by **Jinho Jang** — eric@osaurus.ai |
|
|
