Image-Text-to-Text
MLX
Safetensors
inkling2_loop
apple-silicon
Mixture of Experts
dspark
speculative-decoding
conversational
2-bit
Instructions to use netprojects/Inkling-2-Mini-MLX-2.5bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use netprojects/Inkling-2-Mini-MLX-2.5bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("netprojects/Inkling-2-Mini-MLX-2.5bit") config = load_config("netprojects/Inkling-2-Mini-MLX-2.5bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use netprojects/Inkling-2-Mini-MLX-2.5bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-2.5bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "netprojects/Inkling-2-Mini-MLX-2.5bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use netprojects/Inkling-2-Mini-MLX-2.5bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-2.5bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default netprojects/Inkling-2-Mini-MLX-2.5bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use netprojects/Inkling-2-Mini-MLX-2.5bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-2.5bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "netprojects/Inkling-2-Mini-MLX-2.5bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 9,084 Bytes
cf3da60 8cfdaee cf3da60 d95f790 cf3da60 d95f790 cf3da60 d95f790 cf3da60 d95f790 cf3da60 d95f790 cf3da60 d95f790 cf3da60 6e2cc37 cf3da60 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 | ---
license: apache-2.0
license_link: https://www.apache.org/licenses/LICENSE-2.0
base_model: thinkingmachines/Inkling-2-Mini
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
tags:
- mlx
- apple-silicon
- inkling2_loop
- moe
- dspark
- speculative-decoding
---
# Inkling 2 Mini for Apple Silicon (MLX, 2.5-bit, + DSpark)
[thinkingmachines/Inkling-2-Mini](https://huggingface.co/thinkingmachines/Inkling-2-Mini) (revision `6e7c824`) for MLX on
Apple Silicon, with its DSpark speculative-decoding draft bundled (`mtp.safetensors`) and its vision encoder included.
The routed experts are re-quantized from their released NVFP4 / MXFP8 values to 2-bit (affine, group 64), with 3-bit in layers 0–1 and in layers 20–23 (the layers Thinking Machines kept at MXFP8). Attention, embeddings, the output layer and the vision encoder are 4-bit. Size: 10.8 GB plus the 0.64 GB draft.
Inkling 2 Mini is Thinking Machines' looped mixture-of-experts model: 26.4B parameters, about 3.4B active per token,
24 layers of which the middle twelve run three times (48 layer passes per token), text and image input, up to 1M tokens of context.
## Builds
| Build | Size | Same next token as Thinking Machines' code | KL vs their code | GSM8K (50 questions) | Peak memory, 8K-token prompt |
|---|---|---|---|---|---|
| [netprojects/Inkling-2-Mini-MLX-4-8bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-4-8bit) (recommended) | 19.5 GB + 0.64 GB draft | 98.6% | 0.0018 | 47/50 | 27.8 GB |
| [netprojects/Inkling-2-Mini-MLX-3bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-3bit) | 14.1 GB + 0.64 GB draft | 95.7% | 0.0421 | 48/50 | 21.2 GB |
| [netprojects/Inkling-2-Mini-MLX-2.5bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-2.5bit) | 10.8 GB + 0.64 GB draft | 87.3% | 0.1904 | 45/50 | 18.2 GB |
| [netprojects/Inkling-2-Mini-MLX-4bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-4bit) | 16.5 GB + 0.64 GB draft | 95.9% | 0.0286 | 47/50 | 23.5 GB |
"Thinking Machines' code" is their Transformers implementation of Inkling 2 Mini, run on the released weights; the
columns are explained under "What was checked".
## Install
Inkling 2 Mini support is not in a released MLX-VLM yet. Install MLX-VLM with it from [internet-projects/inkling2-mlx](https://github.com/internet-projects/inkling2-mlx)
(MLX-VLM plus Inkling 2 Mini support; Python 3.10 or newer):
```bash
pip install "git+https://github.com/internet-projects/inkling2-mlx@v0.1.0" torch torchvision
```
PyTorch and torchvision are only used on the CPU by the image processor.
## Use
```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("netprojects/Inkling-2-Mini-MLX-2.5bit")
prompt = apply_chat_template(processor, model.config, "Explain in two sentences why the sky is blue.",
reasoning_effort="low")
print(generate(model, processor, prompt, max_tokens=2048, temperature=0.0).text)
```
`reasoning_effort` is part of Inkling's chat template: `none`, `minimal`, `low`, `medium`, `high` (the default), `xhigh`,
`max`, or a number from 0.0 to 0.99. Thinking Machines notes that the model can think at length even at `none` and can get
stuck in long reasoning loops, so set `max_tokens`.
With an image:
```python
prompt = apply_chat_template(processor, model.config, "What error does this terminal show?", num_images=1, reasoning_effort="low")
print(generate(model, processor, prompt, image=["screenshot.png"], max_tokens=2048).text)
```
With the DSpark draft (greedy output stays the model's own, up to near-ties):
```python
from mlx_vlm.speculative.drafters import load_drafter
draft, kind = load_drafter("netprojects/Inkling-2-Mini-MLX-2.5bit")
print(generate(model, processor, prompt, max_tokens=2048, draft_model=draft, draft_kind=kind,
draft_block_size=4).text)
```
Server: `mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-2.5bit --draft-model netprojects/Inkling-2-Mini-MLX-2.5bit --draft-kind dflash --draft-block-size 4`.
OpenAI-compatible server, with tool calls and the thinking returned separately (`reasoning_content`):
```bash
mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-2.5bit --port 8000
```
## Speed
MacBook Pro M5 Max (128 GB), idle, 21 November 2026; median of 3 runs. Decode: 5 prompts (math, code, chat) × 256 tokens, greedy, reasoning `none`. DSpark: the bundled draft at its best block size.
| Build | Prompt processing, 1K / 8K / 32K tokens (tok/s) | Generation (tok/s) | Generation with DSpark (tok/s) |
|---|---|---|---|
| 4-8bit | 1752 / 1658 / 1208 | 36.6 | 47.1 (×1.29, block 3) |
| 4bit | 1535 / 1412 / 1104 | 43.6 | 70.9 (×1.63, block 4) |
| 3bit | 1386 / 1351 / 1075 | 44.4 | 75.2 (×1.69, block 4) |
| 2.5bit (this build) | 1482 / 1384 / 1100 | 45.7 | 83.9 (×1.84, block 4) |
The draft's default block size (8) is slower on this model; use 3 or 4 as below.
## Thinking Machines' results
From the [Inkling 2 Mini card](https://huggingface.co/thinkingmachines/Inkling-2-Mini), for the original model (their
harnesses and settings; a selection of their table):
| Benchmark (Thinking Machines' figures) | Inkling 2 Mini | Qwen3.8-27B | Gemma 4 31B |
|---|---|---|---|
| Artificial Analysis Intelligence Index v5.0 | 20.1 | 20.1 | 4.4 |
| SWE-bench Verified (bash only) | 79.2% | 77.8% | 46.1% |
| SWE-bench Pro v2 Hard (mini-swe-agent) | 77.9% | 73.8% | 14.2% |
| Terminal-Bench 2.1 (pi) | 71.6% | 73.0% | 43.4% |
| BrowseComp (with compaction) | 79.0% | 61.6% | 37.0% |
| MCP Atlas | 82.7% | 77.3% | 54.2% |
| MMMU Pro | 74.9% | 76.3% | 73.4% |
| OmniDocBench v1.6 | 92.8% | 91.5% | 82.5% |
| LongBench v2 | 66.9% | 52.3% | 52.9% |
| ARC-AGI v1 / v2 | 84.5 / 49.3% | 87.5 / 42.4% | – |
## What was checked
On a MacBook Pro M5 Max (128 GB), 21 November 2026, with MLX 0.32.4 and [inkling2-mlx](https://github.com/internet-projects/inkling2-mlx) v0.1.0.
- **Against Thinking Machines' code.** Their Transformers implementation (pull request #51050) ran on the CPU with the
released weights, on 12 short chat texts (488 answer positions). This build gave the same next token at
87.3% of positions (mean KL 0.1904), and its greedy answers to 3 prompts matched theirs for the first 4, 9, 4 of 24 tokens. Against our 4/8-bit build: 87.3% same next token, mean KL 0.1869.
- **GSM8K, 50 fixed test questions** (reasoning `none`, greedy): 45/50.
- **Tool calling** (reasoning `none`): an agent prompt with `bash` and `read` tools produced a well-formed call in Inkling's
format, which `mlx_vlm.server` returns as OpenAI `tool_calls`.
- **Coding:** 3/3 short Python tasks passed their asserts. **Thinking** (reasoning `low`): 2/2
questions answered correctly after one closed thinking block. **Vision:** named the red square and blue circle of a test image.
- **Memory:** peak 17.2 GB with a 4K-token prompt and 18.2 GB with an 8K-token prompt (mlx-vlm defaults). With a smaller prefill step the peak drops: 12.8 / 13.1 GB at `prefill_step_size=512` and 11.6 / 11.9 GB at `prefill_step_size=256` (4K / 8K-token prompts, plus 0.64 GB if the DSpark draft is loaded), which fits the default GPU memory limit of a 24 GB Mac: pass `prefill_step_size=256` to `generate`, or `--prefill-step-size 256` to `mlx_vlm.server`.
- **DSpark:** the bundled draft loads with this build; greedy output follows the model's own except at near-ties (checked: divergences only where the top two tokens were within 0.2 logits). Speeds are under "Speed".
Running on a 16 GB Mac: this is the smallest build, but even with a small prefill step it needs about 11.1-11.9 GB (4K-8K-token prompt, draft not loaded), a little above the default GPU memory limit of a 16 GB Mac (about 10.7 GB). Raise the limit until the next restart with `sudo sysctl iogpu.wired_limit_mb=12288`, which leaves about 4 GB for macOS; close other apps, do not load the DSpark draft, pass `prefill_step_size=128` (or 256) to `generate` or `--prefill-step-size 128` to `mlx_vlm.server`, and keep prompts within about 8K tokens. Measured peaks without the draft: 4K-token prompt 11.1 GB (step 128) / 11.6 GB (256) / 12.8 GB (512); 8K-token prompt 11.3 / 11.9 / 13.1 GB. On a 24 GB Mac it fits the default limit with `prefill_step_size=512` or lower, with the draft loaded.
Known limitation of the current server: when the model calls a tool, `content` can also carry a short sentence of its
reasoning and the tool name.
## License and credits
Apache-2.0, as Inkling 2 Mini. Use of the model is also governed by Thinking Machines'
[Model Acceptable Use Policy](https://thinkingmachines.ai/model-acceptable-use-policy/). `LICENSE` is the Apache-2.0
text and `NOTICE` describes the conversion. Inkling 2 Mini is by Thinking Machines Lab; please cite their work.
Published by [NetProjects](https://netprojects.com) (Internet Projects Ltd). Not affiliated with Thinking Machines.
More MLX builds from NetProjects: [huggingface.co/netprojects](https://huggingface.co/netprojects) · [netprojects.com](https://netprojects.com)
|