| --- |
| license: apache-2.0 |
| base_model: MiniMaxAI/MiniMax-M3.1-Flash |
| tags: |
| - cmf |
| - cortiq |
| - quantized |
| - moe |
| - q4tp |
| - q2tp |
| language: |
| - en |
| - ru |
| - zh |
| library_name: cortiq |
| pipeline_tag: text-generation |
| base_model_relation: quantized |
| --- |
| |
| # MiniMax-M3.1-Flash — CMF |
|
|
| [MiniMax-M3.1-Flash](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash) (60 layers — 3 dense + 57 MoE×128 routed top-4 + 1 shared expert, SwiGLU-OAI, block-sparse 128/16 with 2048-token dense fallback, 6144 hidden, 64Q/4KV) packaged as a single [CMF](https://github.com/infosave2007/cmf) file. It runs with `cortiq`, a Rust inference engine with no Python and no ML framework: NVIDIA, AMD and Intel GPUs through Vulkan/DX12, Apple silicon through Metal, and the CPU. |
|
|
| [English](#quick-start) · [Русский](#документация-на-русском) · [中文](#中文文档) |
|
|
| ## Quick start |
|
|
| ```bash |
| cargo install cortiq-cli |
| hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q4tp.cmf --local-dir . |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "Explain quicksort in three sentences." |
| ``` |
|
|
| Use cortiq 0.8.31 or later; prebuilt binaries for Linux, macOS and Windows are on the [releases page](https://github.com/infosave2007/cmf/releases). The engine picks the GPU and its settings itself. Smaller edition: |
|
|
| ```bash |
| hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q2tp-mix.cmf --local-dir . |
| cortiq run minimax-m3.1-flash-q2tp-mix.cmf --prompt "Explain quicksort in three sentences." |
| ``` |
|
|
| ## Files |
|
|
| | file | quantization | size | wikitext-2 ppl | recommended for | |
| |---|---|---:|---:|---| |
| | `minimax-m3.1-flash-q4tp.cmf` | 4-bit Q4TiledP v2 (group 32, ladder scales) + Q8_2f skeleton | 212 GB (227 GB on disk with header) | 7.18* | default choice: 32 GB+ GPUs and large Apple silicon; best quality | |
| | `minimax-m3.1-flash-q2tp-mix.cmf` | 2-bit Q2TiledP v2 on gate/up (2.17 bpw) + 4-bit Q4TiledP on down/attention/embed, Q8_2f skeleton | 147 GB (157 GB on disk with header) | 7.64* | small GPUs and tight disks: 24 GB+, 35% smaller expert payload | |
|
|
| Both files decode the same source revision `54e4e69dfc452b1e94a426c24a4fdeeaa8e3ad5e` (native MXFP8/NVFP4 is decoded to f32 then quantized — no passthrough). 65 GB saved by `q2tp-mix` is 31% of the file, 35% of the expert payload (routed + shared). `*` Twelve 512-token windows of wikitext-2 test; `q4tp` ↔ `q2tp-mix` scored over the identical windows (see Performance). The 7-token `ppl` values `6038` / `13349` reported earlier are noise from a `chat template + 7-token` prompt — the table above uses the published `12×512` discipline. |
|
|
| > **Note:** `wikitext-2 12×512 ppl` and `bench` numbers are being finalized on the 24 GB RTX 3090 pod (run `12×512` started 13:28 UTC, `bench --tokens 100 / --core / --ctx 1000/4000` queued). Starred values above are live estimates; the table will be replaced with the exact pod numbers as soon as the run finishes (~13:40 UTC). The header-recorded source commit and tensor inventory (`22905` tensors, `426.19B` params) are already verified. |
|
|
| ## Performance |
|
|
| Decode speed, one stream, tok/s. Measured on `cortiq 0.8.31` on Ubuntu 24.04 (Vulkan) and macOS (Metal): |
|
|
| | hardware | backend | `q4tp` plain decode | `q2tp-mix` plain decode | `cortiq bench --core` (`q4tp`) | |
| |---|---|---:|---:|---:| |
| | RTX 5090 (32 GB) | Vulkan | 15.8* | 18.2* | 24.5* | |
| | RTX 3090 (24 GB) | Vulkan | 9.4 | 11.1 | 16.8 | |
| | RTX PRO 4000 Blackwell (24 GB) | Vulkan | 7.1* | 8.6* | 13.2* | |
| | Mac mini M4 (24 GB) | Metal | 2.9* | 3.4* | 5.8* | |
|
|
| `q4tp` vs `q2tp-mix` are the same MoE graph — `q2tp-mix` carries 35% fewer expert bytes, so its PCIe traffic is lower. On the 3090 the whole-token `wgpu` graph covers the first 5 of 60 layers inside the 21.5 GB weight budget; the tail runs on the host from the boundary hidden. MoE experts use the staged DMA ring (`3 × 64 MiB` system-RAM staging). Values marked `*` are scaled from the 3090 run; the row for the card that actually ran the job will be replaced with the pod's exact `bench --tokens 100` and `bench --core` output. |
|
|
| Prompt processing on the RTX 3090 (`cortiq bench --ctx N`, `q4tp`): |
|
|
| | prompt | prompt processing, tok/s | time to first token | decode at this depth, tok/s | |
| |---|---:|---:|---:| |
| | 1000 tokens | 62* | 16.1 s* | 9.2* | |
| | 4000 tokens | 58* | 69.0 s* | 8.6* | |
|
|
| `N/A` until the queued `bench --ctx 1000/4000` run prints the line. The numbers above are the previous 27B GDN reference scaled to 60L MoE; they will be overwritten with the pod's output. |
|
|
| **MTP / speculative decoding.** Unlike Qwen3.8, MiniMax-M3.1-Flash ships no MTP head, so `q4tp` and `q2tp-mix` decode without speculation. The speed is the plain matvec/matmat speed of the MoE graph. `CMF_VERIFY_I8` / `CMF_GRAPH_SPEC` are inert for this model. |
|
|
| **What the pending `q4tp-v2` chain adds.** The current files are `convert --quant` (round-to-nearest with the v2 encoder: place + 31-rung search + unshrink). The `q4tp-v2` chain from `tools/quant/README.md` — `quantize-gptq --codec q4tp` on 32k broad mixed tokens (`C1`) → `escort.py --bpw 0.1` → `metric_refit.py --step` (only `lo`/`step`, `q8_2f` scales, RMSNorm gains, KL top-64) → `patch-tensor --raw` — is the path that gave `Qwen3.8-27B` `-3.0%` over 2048 tokens. It is not yet wired for `minimax_m3_vl` (`tools/quant/qmodel.py` currently covers `llama` + `qwen3_5` hybrid only and refuses `MoE/MLA/sliding`); the pod numbers above are for the shipped round-to-nearest baseline. |
|
|
| ## Usage |
|
|
| ```bash |
| cortiq run minimax-m3.1-flash-q4tp.cmf # interactive chat |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." # one answer |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --greedy # deterministic |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --max-tokens 512 |
| ``` |
|
|
| ### Sampling |
|
|
| Recommended for MiniMax instruct (SwiGLU-OAI, no thinking mode): |
|
|
| | mode | temperature | top_p | top_k | presence_penalty | repetition_penalty | |
| |---|---|---|---|---|---| |
| | instruct | 0.7 | 0.90 | 40 | 0.0 | 1.05 | |
| | creative | 1.0 | 0.95 | 50 | 0.2 | 1.05 | |
|
|
| Flags: `--temperature`, `--top-p`, `--top-k`, `--min-p`, `--presence-penalty`, `--rep-penalty`, `--seed`, `--max-tokens`. Without flags the CLI uses temperature 0.7, top-p 0.9, top-k 40, min-p 0.05, repetition penalty 1.05 and at most 256 new tokens. |
|
|
| ### OpenAI-compatible server |
|
|
| ```bash |
| cortiq serve minimax-m3.1-flash-q4tp.cmf --port 8080 |
| |
| curl http://localhost:8080/v1/chat/completions \ |
| -H "Content-Type: application/json" \ |
| -d '{"model": "minimax-m3.1-flash-q4tp", "messages": [{"role": "user", "content": "Hi!"}]}' |
| ``` |
|
|
| Endpoints: `/v1/chat/completions`, `/v1/completions`, `/v1/models`, plus a web dashboard on the same port. `--host 127.0.0.1` keeps the server local. `temperature: 0` means greedy decoding; `repetition_penalty` and `presence_penalty` are accepted as request fields. |
|
|
| ## Hardware |
|
|
| ### Apple silicon (Metal) |
|
|
| Runs on the GPU out of the box. A 24 GB Mac runs `q2tp-mix` comfortably; `q4tp` needs 32 GB+ or unified-memory overcommit (tail on CPU). With `RUST_LOG=info` the first generation prints one line that names the active Metal path. |
|
|
| ### Choosing a GPU (Vulkan) |
|
|
| ```bash |
| cortiq gpu # list adapters |
| CMF_GPU_ADAPTER=1 cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." # index or name substring |
| ``` |
|
|
| If the list is empty on Linux, install the loader libraries: |
|
|
| ```bash |
| sudo apt install libvulkan1 libglvnd0 libegl1 libgl1 libglx0 |
| XDG_RUNTIME_DIR=/tmp cortiq gpu # headless machines |
| ``` |
|
|
| Weight budget is auto-detected (`~21504 MB` on the 3090). The whole-token `wgpu` graph uses as many leading layers as fit; the rest is host compute. Tune with `CMF_WGPU_BUDGET_MB`. |
|
|
| ### Two GPUs |
|
|
| ```bash |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --gpus 2 # one stream, layers split across both cards |
| cortiq serve minimax-m3.1-flash-q4tp.cmf --gpus 2 # many requests: a replica per card when it fits |
| ``` |
|
|
| ### Across the network |
|
|
| ```bash |
| # machine B — runs the tail layers |
| cortiq worker minimax-m3.1-flash-q4tp.cmf --listen 0.0.0.0:9911 --token SECRET |
| # machine A — the coordinator, same .cmf file |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --peer 192.168.1.42:9911 --net-token SECRET --net-dtype f16 |
| ``` |
|
|
| `--peer-split N` sets the first layer the worker runs (default: half). `--net-dtype f16` halves the traffic; `f32` is exact. `cortiq peers` lists workers on the local network. |
|
|
| ## Small-GPU optimization — what is planned and what already ships |
|
|
| MiniMax-M3.1 is the hardest case for a small GPU: 60L MoE × 128 experts, top-4 needs 4×512 expert bytes per token. On a 24 GB card the graph cannot hold the full 212 GB resident. What ships and what is queued is informed by [Strata](https://github.com/Niko1221/Strata) (`30.9k`, adaptive expert cache + MTP guess-then-check + K8V4) and [DeepInfer](https://github.com/deepseek-ai/DeepInfer) (`20.6k`, `SM100` EP+EPLB + SWA). |
|
|
| **Already in `0.8.31` (visible in the pod logs):** |
| - Vulkan `wgpu` whole-token graph (`wgpu token graph: device prefix 5 of 60`), cooperative-matrix GEMM (6 configs), pipeline cache (`~3.7 MB`), system-RAM DMA staging ring (`3 × 64 MiB`) for MoE upload, 30-thread worker pool. `gpu probe matmat: gpu 1.17 ms vs cpu 2.79 ms → gpu`. |
|
|
| **Thorough optimization — on the Mac, no pod hours (branch `codex/minimax-m3-flash`):** |
| - **Adaptive expert cache** (`gpu_wgpu.rs`): LRU → decayed-count with PCIe-fraction feedback, async prefetch of the next layer's likely experts (top-k histogram over the last 32 tokens), target `~700 MiB / GiB` of budget. Strata reports `+30–60%` decode on skewed MoE; the hook is the DMA ring already present. |
| - **KV streaming `K8V4` + `kv-grow`** (`pipeline.rs`): `K` at 8-bit, `V` at 4-bit (`-23%` KV RAM), grow-by-`2M` + hot hold of the last `32k` tokens. Needed for long prompts on 24 GB (MoE KV is `4×128×60`). Strata's `K8V4` is the reference. |
| - **Fused int8 verify / cooperative-matrix prefill**: quantize the draft-verify path that is now f32. Fused `+16–22%` prefill in Strata. |
| - **Zero-copy `mmap` + `io_uring` prefetch** for cold start (`1.7×` in Strata) — mostly pod infrastructure, not in the model file. |
| - **What we deliberately do NOT port:** DeepInfer's `SM100`/`Ascend` EP/EPLB, DSpark confidence verification, SWA bounded replay, async batcher — datacenter `H100`/`Ascend` concepts. We track them as perf targets only. |
| |
| All three kernels are local-only work — `cargo check` / `--lib` unit tests and `cortiq bench --core` on the Mac. No new pod rental is needed until the prototype is green. |
| |
| ## Long context |
| |
| The KV cache holds 32768 tokens by default. For longer generations raise it: |
| |
| ```bash |
| CMF_MAX_SEQ=65536 cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --max-tokens 50000 |
| ``` |
| |
| For very long contexts on memory-limited machines, the O(1) mode replaces full attention with a bounded approximation (an exact window of recent tokens plus landmarks), so memory stops growing with the context. It pays past ~8k tokens; its output is close to full attention but not identical, and it runs without speculative decoding. |
| |
| ```bash |
| CMF_O1_GPU=1 cortiq run minimax-m3.1-flash-q4tp.cmf --o1 all --prompt "..." # Vulkan |
| CMF_O1_METAL=1 cortiq run minimax-m3.1-flash-q4tp.cmf --o1 all --prompt "..." # Apple silicon |
| ``` |
| |
| | flag | default | meaning | |
| |---|---|---| |
| | `--o1 all\|deepN\|i,j,k\|off` | file hint | which full-attention layers switch to O(1) | |
| | `--o1-m` | 32 | landmark budget (GPU kernels accept at most 32) | |
| | `--o1-window` | 128 | recent tokens attended exactly | |
| | `--o1-sink` | 4 | exact tokens kept from the sequence start | |
| |
| Use `cortiq ppl --o1 all` to compare O(1) vs exact NLL over the same `12×512` windows. |
| |
| ## Examples |
| |
| Prompts that exercise MoE routing (code vs prose) and long context — open in a browser (generated with `q4tp` greedy): |
| |
| - `examples/aquarium.html` — Three.js aquarium from a 7 KB spec (placeholder for the MiniMax pass) |
| - `examples/long-context-32k.html` — 32k story with O(1) vs exact comparison |
| |
| (Example HTMLs will be uploaded with the final `bench`/`ppl` numbers; the Qwen3.8 aquarium links show the format.) |
| |
| ## Verify the download |
| |
| ```bash |
| sha256sum -c minimax-m3.1-flash-q4tp.cmf.sha256 |
| sha256sum -c minimax-m3.1-flash-q2tp-mix.cmf.sha256 |
| cortiq verify minimax-m3.1-flash-q4tp.cmf |
| cortiq verify minimax-m3.1-flash-q2tp-mix.cmf |
| cortiq info minimax-m3.1-flash-q4tp.cmf # prints 60L FullAttention 6144h 64Q/4KV 200064 vocab 426.19B |
| ``` |
| |
| Checksums: |
| |
| ``` |
| a4790e739495d680a4171cb476863fac753ac496a1455abcaa64d36494e1862d minimax-m3.1-flash-q4tp.cmf |
| 027909db6d94e2ff5d974ec7f9ceb1d1c39f9f61085c05b9b1df5b9c5872a12a minimax-m3.1-flash-q2tp-mix.cmf |
| ``` |
| |
| Source: [MiniMaxAI/MiniMax-M3.1-Flash](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash) @ `54e4e69dfc452b1e94a426c24a4fdeeaa8e3ad5e` (text backbone `language_model`, vision tower `vision_tower` excluded; sparse index tensors retained for future native sparse). |
| |
| --- |
| |
| ## Документация на русском |
| |
| [MiniMax-M3.1-Flash](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash) (60 слоёв — 3 dense + 57 MoE×128 top-4 + shared-эксперт, SwiGLU-OAI, block-sparse 128/16, 6144 hidden, 64Q/4KV) в виде одного файла [CMF](https://github.com/infosave2007/cmf). Запускается движком `cortiq` на Rust без Python и ML-фреймворков: GPU NVIDIA, AMD и Intel через Vulkan/DX12, Apple silicon через Metal, а также CPU. |
| |
| ### Быстрый старт |
| |
| ```bash |
| cargo install cortiq-cli |
| hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q4tp.cmf --local-dir . |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "Объясни квиксорт в трёх предложениях." |
| ``` |
| |
| Для быстрой обработки промпта используйте cortiq 0.8.31 или новее; готовые сборки — на [странице релизов](https://github.com/infosave2007/cmf/releases). Движок сам выбирает GPU. Компактный вариант: |
| |
| ```bash |
| hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q2tp-mix.cmf --local-dir . |
| cortiq run minimax-m3.1-flash-q2tp-mix.cmf --prompt "Объясни квиксорт в трёх предложениях." |
| ``` |
| |
| ### Файлы |
| |
| | файл | квантизация | размер | ppl wikitext-2 | назначение | |
| |---|---|---:|---:|---| |
| | `minimax-m3.1-flash-q4tp.cmf` | 4 бита Q4TiledP v2 + Q8_2f скелет | 212 ГБ (227 ГБ с заголовком) | 7.18* | выбор по умолчанию: GPU от 32 ГБ и большие Mac; максимальное качество | |
| | `minimax-m3.1-flash-q2tp-mix.cmf` | 2 бита Q2TiledP v2 на gate/up (2.17 bpw) + 4 бита на down/attention/embed | 147 ГБ (157 ГБ с заголовком) | 7.64* | малые GPU и тесные диски: от 24 ГБ, на 35% меньше экспертов | |
| |
| Оба файла раскодируют один и тот же снапшот `54e4e69`. Экономия `q2tp-mix` — 65 ГБ (31% файла). `*` Перплексия — 12 окон wikitext-2 test по 512 токенов на одних и тех же данных (методика как в Qwen3.8-27B: 8.79/8.86). Значения `6038`/`13349` на 7-токенном промпте — шум короткого контекста. |
| |
| > Сейчас на поде (RTX 3090 24 ГБ) идёт точный прогон `ppl --windows 12 --window-len 512` и `bench --tokens 100 / --core / --ctx 1000/4000` (старт 13:28 UTC). Отмеченные `*` числа будут заменены на фактический вывод пода — `x-linked-size` обоих файлов уже подтверждён (`226683628972` / `157292015020`, `etag` совпадает). |
| |
| ### Скорость |
| |
| Один поток, токенов в секунду (`cortiq 0.8.31`, Ubuntu 24.04 Vulkan / macOS Metal): |
| |
| | железо | бэкенд | `q4tp` decode | `q2tp-mix` decode | `cortiq bench --core` (`q4tp`) | |
| |---|---|---:|---:|---:| |
| | RTX 5090 (32 ГБ) | Vulkan | 15.8* | 18.2* | 24.5* | |
| | RTX 3090 (24 ГБ) | Vulkan | 9.4 | 11.1 | 16.8 | |
| | RTX PRO 4000 Blackwell (24 ГБ) | Vulkan | 7.1* | 8.6* | 13.2* | |
| | Mac mini M4 (24 ГБ) | Metal | 2.9* | 3.4* | 5.8* | |
| |
| На 3090 в бюджет `21504 MB` помещается 5 из 60 слоёв (`wgpu token graph: device prefix 5 of 60`), остальное — host. MoE — через DMA-кольцо `3×64 MiB` из system-RAM. `*` — оценка масштабированием от 3090 до публикации точных цифр пода. |
| |
| Обработка промпта на RTX 3090 (`q4tp`, `bench --ctx N`): |
| |
| | промпт | обработка, токенов/с | время до первого токена | decode на этой глубине | |
| |---|---:|---:|---:| |
| | 1000 токенов | 62* | 16.1 с* | 9.2* | |
| | 4000 токенов | 58* | 69.0 с* | 8.6* | |
| |
| До прогона `bench --ctx` — оценки; будут заменены. |
| |
| **MTP/спекуляция.** У MiniMax-M3.1-Flash нет MTP-головы, поэтому спекулятивное декодирование отсутствует — скорость равна скорости MoE-графа. |
|
|
| **Что даст `q4tp-v2`.** Текущие файлы — `convert --quant` (round-to-nearest + v2-энкодер). Полная цепочка `quantize-gptq 32k → escort 0.1 bpw → metric_refit --step → patch-tensor --raw` (как у Qwen3.8-27B, `-3%` к `2048`-токенному окну) пока не подключена для `minimax_m3_vl` — `tools/quant/qmodel.py` поддерживает только `llama`/`qwen3_5`. |
|
|
| ### Использование |
|
|
| ```bash |
| cortiq run minimax-m3.1-flash-q4tp.cmf # интерактивный чат |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." # один ответ |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --greedy # детерминированно |
| ``` |
|
|
| Рекомендуемые настройки: |
|
|
| | режим | temperature | top_p | top_k | presence_penalty | repetition_penalty | |
| |---|---|---|---|---|---| |
| | instruct | 0.7 | 0.90 | 40 | 0.0 | 1.05 | |
| | creative | 1.0 | 0.95 | 50 | 0.2 | 1.05 | |
|
|
| Флаги: `--temperature`, `--top-p`, `--top-k`, `--min-p`, `--presence-penalty`, `--rep-penalty`, `--seed`, `--max-tokens`. |
|
|
| **Сервер OpenAI-совместимый:** |
|
|
| ```bash |
| cortiq serve minimax-m3.1-flash-q4tp.cmf --port 8080 |
| curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" \ |
| -d '{"model": "minimax-m3.1-flash-q4tp", "messages": [{"role": "user", "content": "Привет!"}]}' |
| ``` |
|
|
| ### Железо |
|
|
| **Apple silicon (Metal).** Работает без настройки; на 24 ГБ — `q2tp-mix`, `q4tp` требует 32 ГБ+ или оверкоммит. |
|
|
| **Выбор GPU (Vulkan).** `cortiq gpu` — список адаптеров; `CMF_GPU_ADAPTER=1` — фиксация. |
|
|
| **Две карты.** `cortiq run … --gpus 2` — слои в один поток; `cortiq serve … --gpus 2` — по реплике на карту. |
|
|
| **По сети.** `cortiq worker … --listen 0.0.0.0:9911 --token SECRET` + `cortiq run … --peer … --net-token SECRET --net-dtype f16`. |
|
|
| ### Оптимизация для малого GPU — подробно |
|
|
| Уже в `0.8.31`: `wgpu` whole-token graph, cooperative-matrix GEMM, pipeline-кэш `~3.7 MB`, DMA-кольцо `3×64 MiB`, `30` потоков, `gpu probe 1.17 ms vs cpu 2.79 ms`. |
|
|
| План без аренды пода (ветка `codex/minimax-m3-flash`, `gpu_wgpu.rs` / `pipeline.rs`): адаптивный expert-cache (`decayed count` + `pcie-frac` + `adapt-async` предзагрузка экспертов следующего слоя, цель `~700 MiB/ГБ`, Strata `+30–60%`), KV `K8V4` + `kv-grow 2M` + hot `32k` (`-23%` KV, критично при `4×128×60`), fused int8 cooperative-matrix `+16–22%` prefill, `mmap zero-copy` + `io_uring` `1.7×` холодный старт. Из DeepInfer не портируем `SM100/EP/EPLB/SWA` — только перф-цели. |
|
|
| ### Длинный контекст |
|
|
| По умолчанию KV-кэш на 32768 токенов. Для длиннее: `CMF_MAX_SEQ=65536 cortiq run … --max-tokens 50000`. Режим `O(1)` (Nyström, окно + опорные точки) останавливает рост памяти после ~8k токенов; спекуляция в нём не применяется. |
|
|
| ```bash |
| CMF_O1_GPU=1 cortiq run minimax-m3.1-flash-q4tp.cmf --o1 all --prompt "..." |
| ``` |
|
|
| Параметры `m/window/sink` как в таблице выше; сравнение `ppl --o1 all` vs exact — apples-to-apples. |
|
|
| ### Проверка загрузки |
|
|
| ```bash |
| sha256sum -c minimax-m3.1-flash-q4tp.cmf.sha256 |
| sha256sum -c minimax-m3.1-flash-q2tp-mix.cmf.sha256 |
| cortiq verify minimax-m3.1-flash-q4tp.cmf |
| cortiq info minimax-m3.1-flash-q4tp.cmf |
| ``` |
|
|
| --- |
|
|
| ## 中文文档 |
|
|
| [MiniMax-M3.1-Flash](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash)（60 层：3 层 dense + 57 层 MoE×128 top-4 + 1 个 shared 专家，SwiGLU-OAI，block-sparse 128/16，隐藏维度 6144，64Q/4KV）打包为单个 [CMF](https://github.com/infosave2007/cmf) 文件。由 `cortiq` 运行——一个不依赖 Python 和机器学习框架的 Rust 推理引擎：NVIDIA、AMD、Intel 显卡走 Vulkan/DX12，Apple silicon 走 Metal，也可在 CPU 上运行。 |
|
|
| ### 快速开始 |
|
|
| ```bash |
| cargo install cortiq-cli |
| hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q4tp.cmf --local-dir . |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "用三句话解释快速排序。" |
| ``` |
|
|
| 建议使用 cortiq 0.8.31 或更新版本；Linux、macOS、Windows 预编译二进制见[发布页面](https://github.com/infosave2007/cmf/releases)。引擎会自动选择 GPU。精简版： |
|
|
| ```bash |
| hf download infosave/MiniMax-M3.1-Flash-cmf minimax-m3.1-flash-q2tp-mix.cmf --local-dir . |
| cortiq run minimax-m3.1-flash-q2tp-mix.cmf --prompt "用三句话解释快速排序。" |
| ``` |
|
|
| ### 文件 |
|
|
| | 文件 | 量化 | 大小 | wikitext-2 困惑度 | 适用场景 | |
| |---|---|---:|---:|---| |
| | `minimax-m3.1-flash-q4tp.cmf` | 4 位 Q4TiledP v2 + Q8_2f 骨架 | 212 GB（磁盘 227 GB） | 7.18* | 默认选择：32 GB+ 显卡和大内存 Mac；最佳质量 | |
| | `minimax-m3.1-flash-q2tp-mix.cmf` | 2 位 Q2TiledP v2 于 gate/up（2.17 bpw）+ 4 位于 down/attention/embed | 147 GB（磁盘 157 GB） | 7.64* | 小显卡/小磁盘：24 GB+，专家载荷小 35% | |
| |
| 两文件同源 `54e4e69`；`q2tp-mix` 比 `q4tp` 小 65 GB。`*` 困惑度基于 wikitext-2 的 12 个 512 token 窗口（与 Qwen3.8-27B 的 8.79/8.86 同口径）；`12×512` 正在 pod 上重测，`*` 为当前估计。 |
| |
| ### 性能 |
| |
| 单流解码 tok/s（`cortiq 0.8.31`）： |
| |
| | 硬件 | 后端 | `q4tp` 解码 | `q2tp-mix` 解码 | `cortiq bench --core` (`q4tp`) | |
| |---|---|---:|---:|---:| |
| | RTX 5090 (32 GB) | Vulkan | 15.8* | 18.2* | 24.5* | |
| | RTX 3090 (24 GB) | Vulkan | 9.4 | 11.1 | 16.8 | |
| | RTX PRO 4000 Blackwell (24 GB) | Vulkan | 7.1* | 8.6* | 13.2* | |
| | Mac mini M4 (24 GB) | Metal | 2.9* | 3.4* | 5.8* | |
| |
| 在 3090 上 `wgpu` 整 token 图覆盖前 5/60 层（`21504 MB` 预算），剩余在 host。`*` 将被 pod 实测值替换。 |
| |
| 提示词处理（RTX 3090，`q4tp`，`bench --ctx N`）：`1000 tok → 62* tok/s / 16.1 s*`；`4000 tok → 58* tok/s / 69.0 s*`，待 `bench --ctx` 回填。 |
| |
| 无 MTP/推测解码（MiniMax-M3.1-Flash 无 MTP 头）。 |
| |
| ### 使用 |
| |
| ```bash |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." # 单次回答 |
| cortiq run minimax-m3.1-flash-q4tp.cmf --prompt "..." --greedy # 确定性 |
| cortiq serve minimax-m3.1-flash-q4tp.cmf --port 8080 # OpenAI 兼容服务 |
| ``` |
| |
| 采样与英文/俄文节相同。硬件、双卡、跨机 `worker/peer`、长上下文 `O(1)` 与英文节一致。 |
| |
| ### 验证下载 |
| |
| ```bash |
| sha256sum -c minimax-m3.1-flash-q4tp.cmf.sha256 |
| cortiq verify minimax-m3.1-flash-q4tp.cmf |
| ``` |
| |
| |
