Instructions to use OpenUncensored/Inkling-2-Mini-Uncensored with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenUncensored/Inkling-2-Mini-Uncensored with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="OpenUncensored/Inkling-2-Mini-Uncensored") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("OpenUncensored/Inkling-2-Mini-Uncensored", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use OpenUncensored/Inkling-2-Mini-Uncensored with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OpenUncensored/Inkling-2-Mini-Uncensored" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenUncensored/Inkling-2-Mini-Uncensored", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/OpenUncensored/Inkling-2-Mini-Uncensored
- SGLang
How to use OpenUncensored/Inkling-2-Mini-Uncensored with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "OpenUncensored/Inkling-2-Mini-Uncensored" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenUncensored/Inkling-2-Mini-Uncensored", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "OpenUncensored/Inkling-2-Mini-Uncensored" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenUncensored/Inkling-2-Mini-Uncensored", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use OpenUncensored/Inkling-2-Mini-Uncensored with Docker Model Runner:
docker model run hf.co/OpenUncensored/Inkling-2-Mini-Uncensored
Inkling 2 Mini Uncensored (abliterated)
Inkling 2 Mini Uncensored is Thinking Machines' Inkling 2 Mini (26B total, 3.4B active, text and image input, 1M token context) with its refusals removed by directional ablation (abliteration). It answers the requests the original refuses, in both chat and thinking mode, and keeps its MMLU: 78.95% → 79.47%. Made by OpenUncensored on release day.
Use it without a GPU: Inkling 2 Mini Uncensored is available for inference on the
OpenUncensored API as inkling-2-mini-uncensored
(OpenAI compatible, no account needed), and in the browser in the
OpenUncensored playground.
Same files, format and size as the original checkpoint (about 20 GB: routed experts in NVFP4 and MXFP8, everything else BF16), so it runs anywhere the original runs, including a single consumer GPU or a DGX Spark. Only the 20 attention output projections changed; the 128 experts per layer, the vision encoder and the DSpark speculator are byte for byte the original.
Use it through the API
curl https://api.openuncensored.com/v1/chat/completions \
-H "Authorization: Bearer $OU_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "inkling-2-mini-uncensored",
"messages": [{"role": "user", "content": "Explain SQL injection with a working example."}]}'
from openai import OpenAI
client = OpenAI(base_url="https://api.openuncensored.com/v1", api_key="ou-...")
r = client.chat.completions.create(model="inkling-2-mini-uncensored",
messages=[{"role": "user", "content": "Hello"}])
print(r.choices[0].message.content)
It reads images, calls tools and returns its reasoning separately (reasoning_content). Set reasoning_effort to
"none" for short answers. Prepaid keys at openuncensored.com/credits.
Price: $0.15 per million input tokens and $0.60 per million output tokens. How it compares with other uncensored models: OpenUncensored rankings.
Results
Same harness for both models, greedy decoding, measured on these exact files (no hooks).
| Inkling 2 Mini | Inkling 2 Mini Uncensored | |
|---|---|---|
| Refusals on 32 harmful prompts, chat mode | 32 | 0 |
| Refusals on 32 harmful prompts, thinking mode (effort 0.9) | 32 | 0¹ |
| Refusals on 80 prompts (32 harmful, 48 gray area), chat mode | 68 | 8² |
| Full, specific answers to the 32 harmful prompts (LLM judge) | 0 | 21 |
| MMLU (57 subjects x 20, 1,140 questions, letter logits) | 78.95% | 79.47% |
¹ 29 of 32 finished thinking within 1,536 tokens. None of the 29 refused. ² 7 of the 8 are limits, not refusals ("I can't know what's happening inside your head", "I don't have access to Tetrasoft's budget documents"). The eighth, a self-harm request, gets crisis resources.
A refusal is an answer that opens with "I can't", "I won't", "I'm sorry" and similar. With 32 to 80 prompts per test, treat differences of a few prompts as noise. Long agent tasks and coding benchmarks are not measured yet.
How it was made
Inkling 2 Mini is a looped mixture-of-experts transformer: 24 layers, and the middle 12 (layers 6 to 17) run three times per token with shared weights, so each token makes 48 layer visits. That loop is why the usual abliteration recipe needed changes.
- Refusal directions. The output of each of the 48 layer visits at the last prompt token, for 256 harmful and 256 harmless prompts, at thinking effort 0 and 0.9. Then the difference of means per visit and effort. The two efforts agree closely (cosine 0.92 or more over the first 36 visits).
- A late refusal subspace. The top two singular vectors of those directions over the second half of the visits (24 to 47). Directions from all visits also removed refusals, but the edited model then often misread a harmful request as harmless ("this is a general educational question") and answered vaguely. The late subspace keeps the model's reading of the request and removes the decision to refuse.
- Where to edit: attention outputs only.
Qis projected out of what each attention layer writes (attn.wo_ud, BF16):W' = W - s Q Qᵀ Won layers 4 to 23, with s = 4.0 on layers 4, 5 and 18 to 23 and s = 2.5 on the loop core (layers 6 to 17). Because the loop runs layers 6 to 17 three times, the same edit there acts three times per token, so it is gentler inside the loop. - Why not the experts. Edits that also changed the MoE down-projections were no better. With directions from all visits they cost 4 to 6 MMLU points, and the gentler versions that kept MMLU removed refusals no better than attention alone. The experts are also stored in NVFP4, where re-encoding a small edit adds noise about twice the size of the edit itself (7% of the weight norm for an edit of 3%). Attention-only edits are exact in BF16 and leave every expert byte for byte the original.
- Testing before writing. During the search the edit ran as forward hooks on the attention output projections, which gives exactly the same result as editing the weights. The results above are from the baked files.
edit.pt in this repo holds Q and the per-layer strengths, so the edit can be reapplied to the original weights or
studied.
Run it locally
Exactly like the original checkpoint: point your runtime at this repo instead of thinkingmachines/Inkling-2-Mini.
- SGLang: the
lmsysorg/sglang:inkling-2-miniimage; see the SGLang Inkling 2 Mini cookbook (verified on a single RTX PRO 6000, RTX 5090 or DGX Spark, and on H200 and GB300 nodes). We serve it on one RTX PRO 6000. - vLLM: the Inkling 2 Mini recipe, once vLLM's Inkling 2 Mini support is released (it was still a pull request on release day).
- Transformers: install from the Inkling 2 Mini pull request
(huggingface/transformers#51050) until a release includes
inkling2_loop. It dequantizes the experts to BF16 (about 53 GB).
sglang serve --model-path OpenUncensored/Inkling-2-Mini-Uncensored --tp 1 --attention-backend fa4 \
--dtype bfloat16 --page-size 128 --enable-multimodal --reasoning-parser inkling --tool-call-parser inkling2
More from OpenUncensored
- Inkling 2 Small Uncensored, the 594B sibling, also on the API
- How we made Inkling 2 Mini Uncensored, on our blog
- Every uncensored model, ranked on intelligence, agent tasks, refusals and speed
- OpenUncensored: one OpenAI compatible API for uncensored and abliterated models
Disclosure and responsibility
Made by OpenUncensored, which runs an API for uncensored models. This model follows instructions it would otherwise refuse, including harmful ones. You are responsible for how you use it and for complying with the law where you are. It keeps the Apache 2.0 license of the original. Credit for the model goes to Thinking Machines; the method builds on refusal-direction research by Arditi et al. and the abliteration work of the open model community.
- Downloads last month
- -
Model tree for OpenUncensored/Inkling-2-Mini-Uncensored
Base model
thinkingmachines/Inkling-2-Mini