Uploading now (updated 21 Nov, 15:40 AEDT). The quant passed its held-out quality and refusal gates and the files are going up; the agentic Gauntlet is running and its numbers land here when it finishes.

stage (21 Nov 2026, AEDT) status
abliteration (ARA) + refusal gate done: 6/100 harmful, 1/100 harmless refused
quantisation (~14 GB) done: 13.93 GB GGUF, verified (744 tensors, 0 problems)
held-out KL / top-1 vs the parent done: KL 0.013, top-1 96.3 % vs the abliterated parent
refusal gate on the quant done: 4/100 harmful, 1/100 harmless refused
speed through the real server done: 59.7 tok/s decode, 1,162 tok/s prefill at 8K (RTX 4090)
Gauntlet v3 (agentic coding) running now
upload in progress (model, DSpark drafter, vision projector)

Inkling-2-Mini 路 XQ, abliterated

Thinking Machines' Inkling 2 Mini with the refusal direction edited out of the weights, then requantised with Indra's Mirror's mixed-precision XQ pipeline. A 26B-parameter looped MoE with about 3B active parameters, 128 routed experts at top-8, and a 1M-token context.

Runs on Xolotl, an engine built for this class of model and machine, on top of Strata. The forward graph is ported from llama.cpp (PR #32694 by Pedro Cuenca, MIT); the engine around it is Xolotl's own.

file size 13.93 GB
routed experts mixed per layer, ~4.0 bits/weight on average; 13 of 24 layers keep the official NVFP4 experts bit for bit
dense backbone 8-bit (Q8_0)
arch / tensors inkling2_loop, 24 physical layers, 48 cache slots, 744 tensors

The model

Inkling 2 Mini is a 24-layer decoder-only transformer with sparse MoE feed-forward blocks. Its middle 12 layers (layers 6 to 17) run three times with shared weights, so each token gets 48 layer applications out of 24 layers of weights, and the cache carries 48 slots.

total / active 26B total, about 3B active
loop layers 6 to 17 run 3 passes, 48 cache slots; passes 2 and 3 read the previous pass through per-layer low-rank loop controllers
MoE 128 routed experts, top-8, no shared expert, sigmoid router with a selection bias, SiTU activation
attention sliding window 1024 with a global layer every 6th (5, 11, 17, 23); no RoPE: a learned relative-position bias per head, with log-scaled positions on the global layers
extras short causal convolutions on K, V, the attention output and the FFN output; block attention residuals that mix earlier depths; final logit softcap 30
vocab 200,058
context 1M tokens (the trained key says 1048576)
official precision routed experts NVFP4 on layers 0 to 19, MXFP8 on 20 to 23, everything else BF16

The official checkpoint ships in that mixed precision. This build starts from it rather than from a dequantised copy, so the native FP4 experts can be carried through at their own precision.

How it was made

  1. Abliteration with Heretic-style ARA (Abliteration Rank Adjustment) on the attention output projections of layers 11 to 23, the model's only dense writers into the residual stream. The classic refusal-direction projection does not survive on this model: the routed experts are FP4, and re-encoding an edited expert to FP4 erases most of the edit (measured: only ~13 % of the direction removed). Refusal gate (system "You are a helpful assistant.", thinking off, 96 greedy tokens, a BERT refusal classifier): 6/100 harmful and 1/100 harmless prompts refused, at KL 0.033 against the original on harmless prompts.
  2. Quantisation with XQ: every routed-expert layer gets its own format, picked from each expert's measured output error on held-out text for a given file size; the official NVFP4 experts are kept bit-exact wherever the budget allows, and the dense backbone goes to 8-bit. Statistics come from the abliterated model itself, run through Xolotl's engine. The XQ pipeline itself is private.

Xolotl found and fixed a bug in the llama.cpp loop path along the way: passes 2 and 3 of the loop ran their experts without the per-expert NVFP4 scale when the GGUF keeps NVFP4 experts. The upstream converter dequantises, so upstream never meets that case. It is fixed in Xolotl's build.

Quality

Measured against the parent, which is the official checkpoint converted exactly (NVFP4 and MXFP8 kept) and run on the same engine, hardware and settings.

parent abliterated (before quant) this (XQ)
Gauntlet v3, solved not run running running
Gauntlet v3, score not run running running
held-out top-1 agreement vs the original n/a 95.3 % 94.7 %
held-out KL vs the original n/a 0.028 0.033
held-out KL / top-1 vs the abliterated parent n/a n/a 0.013 / 96.3 %
refusal gate, harmful prompts refused not run 6/100 4/100
refusal gate, harmless prompts refused not run 1/100 1/100

Three columns on purpose: the abliteration and the quant are each checked for damage on their own.

The Gauntlet is Xolotl's agentic coding set: hard tasks run over multiple turns with tools in a sandbox, hidden tests graded after every turn, thinking on, run at the model's recommended sampling over several seeds.

Speed

Measured on an RTX 4090 through Xolotl's server (8K prompt, then 1,024 tokens): 1,162 tok/s prefill, 59.7 tok/s decode. The exact 22.9 GB parent does 1,131 / 57 on the same card, so the quant fits in 14 GB without losing speed.

Decode is far from the card's memory limit (a token reads about 3.4 GB, ~3.4 ms on a 4090, but takes 16.7 ms): it is bound by the number of small GPU kernels per token (48 layer applications). Kernel fusion is the work in progress, aiming at 100+ tok/s.

Landed in Xolotl (21 Nov):

  • Speculative decoding with the model's own DSpark (MTP) drafter (--dspark, experimental): up to 7 drafted tokens per step, with exact rollback of the model's recurrent convolution state. Still being tuned for speed.
  • Quantised K/V cache (--kv-type q8_0|k8v4): the long-context path for 512K on one card.

Next: vision, flash attention with the relative-position bias, and a grouped MoE kernel for speculative verify.

Files

file size what
Inkling-2-Mini-XQ-Uncensored.gguf 13.93 GB the model
mmproj-Inkling-2-Mini-XQ-Uncensored-F16.gguf 0.64 GB the official vision encoder, unchanged, for images (optional; engine support in progress)
Inkling-2-Mini-XQ-Uncensored-DSpark-BF16.gguf 0.64 GB optional: the model's own DSpark (MTP) drafter for speculative decoding (--dspark), converted by unsloth

Quick start

Xolotl's xolotl-inkling engine runs any Inkling-2-Mini GGUF: the upstream converter's, llama.cpp's quants, or this file.

git clone https://github.com/Indras-Mirror/Xolotl.git xolotl && cd xolotl
# build per docs/INSTALL.md, then serve with the Inkling config

Credits

The model is Thinking Machines Lab's, released with open weights under Apache 2.0. The Inkling 2 forward graph and the GGUF converter come from llama.cpp PR #32694 by Pedro Cuenca (MIT), which extends the earlier Inkling 2 work in llama.cpp and builds on ggml and llama.cpp by Georgi Gerganov and the llama.cpp contributors. Xolotl itself is built on Strata by Niko1221.

License

Apache 2.0, the same as the official model. The official card says, in its General Information section: "License: Apache 2.0", with the link to LICENSE.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for IndrasMirror/Inkling-2-Mini-XQ-Uncensored

Quantized
(37)
this model