File size: 8,655 Bytes
57d7fef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
0a45a05
57d7fef
 
 
2279271
57d7fef
 
 
 
 
 
 
 
2279271
 
 
 
57d7fef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2279271
 
57d7fef
 
2279271
 
57d7fef
 
 
 
 
 
2279271
 
 
 
 
 
 
 
 
 
 
 
 
57d7fef
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8285c0d
57d7fef
 
 
 
 
 
 
 
 
2f36859
2279271
57d7fef
c101323
 
57d7fef
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
---
license: apache-2.0
license_link: https://www.apache.org/licenses/LICENSE-2.0
base_model: thinkingmachines/Inkling-2-Mini
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
tags:
  - mlx
  - apple-silicon
  - inkling2_loop
  - moe
  - dspark
  - speculative-decoding
---

# Inkling 2 Mini for Apple Silicon (MLX, 4/8-bit, lossless, + DSpark)

[thinkingmachines/Inkling-2-Mini](https://huggingface.co/thinkingmachines/Inkling-2-Mini) (revision `6e7c824`) for MLX on
Apple Silicon, with its DSpark speculative-decoding draft bundled (`mtp.safetensors`) and its vision encoder included.
The routed experts are kept exactly as Thinking Machines released them: 4-bit NVFP4 in layers 0–19 and 8-bit MXFP8 in layers 20–23, repacked into MLX's native nvfp4 / mxfp8 formats with every value unchanged. Attention, embeddings, the output layer and the vision encoder stay in the original bf16. This is the release itself, in MLX. Size: 19.5 GB plus the 0.64 GB draft.

Inkling 2 Mini is Thinking Machines' looped mixture-of-experts model: 26.4B parameters, about 3.4B active per token,
24 layers of which the middle twelve run three times (48 layer passes per token), text and image input, up to 1M tokens of context.

## Builds

| Build | Size | Same next token as Thinking Machines' code | KL vs their code | GSM8K (50 questions) | Peak memory, 8K-token prompt |
|---|---|---|---|---|---|
| [netprojects/Inkling-2-Mini-MLX-4-8bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-4-8bit) (recommended) | 19.5 GB + 0.64 GB draft | 98.6% | 0.0018 | 47/50 | 27.8 GB |
| [netprojects/Inkling-2-Mini-MLX-3bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-3bit) | 14.1 GB + 0.64 GB draft | 95.7% | 0.0421 | 48/50 | 21.2 GB |
| [netprojects/Inkling-2-Mini-MLX-2.5bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-2.5bit) | 10.8 GB + 0.64 GB draft | 87.3% | 0.1904 | 45/50 | 18.2 GB |
| [netprojects/Inkling-2-Mini-MLX-4bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-4bit) | 16.5 GB + 0.64 GB draft | 95.9% | 0.0286 | 47/50 | 23.5 GB |

"Thinking Machines' code" is their Transformers implementation of Inkling 2 Mini, run on the released weights; the
columns are explained under "What was checked".

## Install

Inkling 2 Mini support is not in a released MLX-VLM yet. Install MLX-VLM with it from [internet-projects/inkling2-mlx](https://github.com/internet-projects/inkling2-mlx)
(MLX-VLM plus Inkling 2 Mini support; Python 3.10 or newer):

```bash
pip install "git+https://github.com/internet-projects/inkling2-mlx@v0.1.0" torch torchvision
```

PyTorch and torchvision are only used on the CPU by the image processor.

## Use

```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("netprojects/Inkling-2-Mini-MLX-4-8bit")
prompt = apply_chat_template(processor, model.config, "Explain in two sentences why the sky is blue.",
                             reasoning_effort="low")
print(generate(model, processor, prompt, max_tokens=2048, temperature=0.0).text)
```

`reasoning_effort` is part of Inkling's chat template: `none`, `minimal`, `low`, `medium`, `high` (the default), `xhigh`,
`max`, or a number from 0.0 to 0.99. Thinking Machines notes that the model can think at length even at `none` and can get
stuck in long reasoning loops, so set `max_tokens`.

With an image:

```python
prompt = apply_chat_template(processor, model.config, "What error does this terminal show?", num_images=1, reasoning_effort="low")
print(generate(model, processor, prompt, image=["screenshot.png"], max_tokens=2048).text)
```

With the DSpark draft (greedy output stays the model's own, up to near-ties):

```python
from mlx_vlm.speculative.drafters import load_drafter
draft, kind = load_drafter("netprojects/Inkling-2-Mini-MLX-4-8bit")
print(generate(model, processor, prompt, max_tokens=2048, draft_model=draft, draft_kind=kind,
               draft_block_size=3).text)
```

Server: `mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-4-8bit --draft-model netprojects/Inkling-2-Mini-MLX-4-8bit --draft-kind dflash --draft-block-size 3`.

OpenAI-compatible server, with tool calls and the thinking returned separately (`reasoning_content`):

```bash
mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-4-8bit --port 8000
```

## Speed

MacBook Pro M5 Max (128 GB), idle, 21 November 2026; median of 3 runs. Decode: 5 prompts (math, code, chat) × 256 tokens, greedy, reasoning `none`. DSpark: the bundled draft at its best block size.

| Build | Prompt processing, 1K / 8K / 32K tokens (tok/s) | Generation (tok/s) | Generation with DSpark (tok/s) |
|---|---|---|---|
| 4-8bit (this build) | 1752 / 1658 / 1208 | 36.6 | 47.1 (×1.29, block 3) |
| 4bit | 1535 / 1412 / 1104 | 43.6 | 70.9 (×1.63, block 4) |
| 3bit | 1386 / 1351 / 1075 | 44.4 | 75.2 (×1.69, block 4) |
| 2.5bit | 1482 / 1384 / 1100 | 45.7 | 83.9 (×1.84, block 4) |

The draft's default block size (8) is slower on this model; use 3 or 4 as below.

## Thinking Machines' results

From the [Inkling 2 Mini card](https://huggingface.co/thinkingmachines/Inkling-2-Mini), for the original model (their
harnesses and settings; a selection of their table):

| Benchmark (Thinking Machines' figures) | Inkling 2 Mini | Qwen3.8-27B | Gemma 4 31B |
|---|---|---|---|
| Artificial Analysis Intelligence Index v5.0 | 20.1 | 20.1 | 4.4 |
| SWE-bench Verified (bash only) | 79.2% | 77.8% | 46.1% |
| SWE-bench Pro v2 Hard (mini-swe-agent) | 77.9% | 73.8% | 14.2% |
| Terminal-Bench 2.1 (pi) | 71.6% | 73.0% | 43.4% |
| BrowseComp (with compaction) | 79.0% | 61.6% | 37.0% |
| MCP Atlas | 82.7% | 77.3% | 54.2% |
| MMMU Pro | 74.9% | 76.3% | 73.4% |
| OmniDocBench v1.6 | 92.8% | 91.5% | 82.5% |
| LongBench v2 | 66.9% | 52.3% | 52.9% |
| ARC-AGI v1 / v2 | 84.5 / 49.3% | 87.5 / 42.4% | – |

## What was checked

On a MacBook Pro M5 Max (128 GB), 21 November 2026, with MLX 0.32.4 and [inkling2-mlx](https://github.com/internet-projects/inkling2-mlx) v0.1.0.

- **Against Thinking Machines' code.** Their Transformers implementation (pull request #51050) ran on the CPU with the
  released weights, on 12 short chat texts (488 answer positions). This build gave the same next token at
  98.6% of positions (mean KL 0.0018), and its greedy answers to 3 prompts matched theirs token for token (24 tokens each). Every tensor of this build is bit-identical to the release as MLX reads it.
- **GSM8K, 50 fixed test questions** (reasoning `none`, greedy): 47/50.
- **Tool calling** (reasoning `none`): an agent prompt with `bash` and `read` tools produced a well-formed call in Inkling's
  format, which `mlx_vlm.server` returns as OpenAI `tool_calls`.
- **Coding:** 3/3 short Python tasks passed their asserts. **Thinking** (reasoning `low`): 2/2
  questions answered correctly after one closed thinking block. **Vision:** named the red square and blue circle of a test image.
- **Memory:** peak 26.9 GB with a 4K-token prompt and 27.8 GB with an 8K-token prompt (mlx-vlm defaults). With a smaller prefill step the peak drops: 22.3 / 22.6 GB at `prefill_step_size=512` and 21.4 / 21.6 GB at `prefill_step_size=256` (4K / 8K-token prompts, plus 0.64 GB if the DSpark draft is loaded), which fits the default GPU memory limit of a 36 GB Mac: pass `prefill_step_size=256` to `generate`, or `--prefill-step-size 256` to `mlx_vlm.server`.
- **DSpark:** the bundled draft loads with this build; greedy output follows the model's own except at near-ties (checked: divergences only where the top two tokens were within 0.2 logits). Speeds are under "Speed".

Precision note: Thinking Machines released the routed experts in 4-bit NVFP4 (8-bit MXFP8 in the last four layers) and no higher-precision weights; MLX builds of this model keep the experts at that precision, so this 4/8-bit build already carries the full released precision.

Known limitation of the current server: when the model calls a tool, `content` can also carry a short sentence of its
reasoning and the tool name.

## License and credits

Apache-2.0, as Inkling 2 Mini. Use of the model is also governed by Thinking Machines'
[Model Acceptable Use Policy](https://thinkingmachines.ai/model-acceptable-use-policy/). `LICENSE` is the Apache-2.0
text and `NOTICE` describes the conversion. Inkling 2 Mini is by Thinking Machines Lab; please cite their work.

Published by [NetProjects](https://netprojects.com) (Internet Projects Ltd). Not affiliated with Thinking Machines.

More MLX builds from NetProjects: [huggingface.co/netprojects](https://huggingface.co/netprojects) · [netprojects.com](https://netprojects.com)