| --- |
| license: apache-2.0 |
| base_model: thinkingmachines/Inkling |
| tags: |
| - abliteration |
| - uncensored |
| - moe |
| - multimodal |
| arxiv: 2406.11717 |
| --- |
| |
| # Inkling Abliterated (scale 2.5 on `main`, scale 3.0 on `scale-3.0`) |
|
|
| <p align="center"> |
| <a href="https://distributed.inc"> |
| <img class="block dark:hidden" src="https://huggingface.co/distributedcognition/dcog-brand-assets/resolve/main/assets/dcog-wordmark-motto.png" alt="Distributed Cognition" width="560"> |
| <img class="hidden dark:block" src="https://huggingface.co/distributedcognition/dcog-brand-assets/resolve/main/assets/dcog-wordmark-motto-dark.png" alt="Distributed Cognition" width="560"> |
| </a> |
| </p> |
| |
| ## Responsible use |
|
|
| This is an uncensored research model. Its built-in refusals were removed permanently at the |
| weight level, so it answers prompts the base model declines, including chemical and |
| biological synthesis, cybercrime, weapons, harassment, and fraud. Use it for red-teaming, |
| offensive-security research, and refusal-rate evaluation. It has no guardrails of its own: |
| if you deploy it, add your own input and output moderation (for example Llama Guard). |
|
|
| ## What this is |
|
|
| Abliterated variant of |
| [`thinkingmachines/Inkling`](https://huggingface.co/thinkingmachines/Inkling) |
| (975B total / 41B active, multimodal MoE, 66 layers). The refusal direction was removed |
| with [norm-preserving biprojected abliteration](https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration) |
| (grimjim 2025), a weight-space refinement of |
| [directional ablation](https://arxiv.org/abs/2406.11717) (Arditi et al. 2024), applied to |
| the attention output projection (`attn.wo_ud`) and the shared-expert down projection |
| (`mlp.shared_experts.shared_w2`) across layers 9–58, using per-layer directions. |
|
|
| Both evaluated scales ship as **git branches** (HF is git+LFS — switching branches is |
| free; identical LFS objects are shared): |
|
|
| | Item | Path / branch | Notes | |
| |---|---|---| |
| | **Scale 2.5** (default) | `main` | Judge refusals 98 → 10/100, GSM8K 65% | |
| | **Scale 3.0** | `scale-3.0` branch | Judge refusals 98 → 16/100, GSM8K 75% | |
|
|
| The repo mirrors the upstream NVFP4 checkpoint (same 33-shard layout, same tokenizer). Only |
| the 100 abliterated tensors differ; everything else is byte-identical to upstream — the |
| attention and shared-expert weights are stored unquantized (BF16) in NVFP4, so the 4-bit |
| routed experts are untouched. `refusal_directions.pt` holds all 66 measured per-layer |
| directions, so you can re-ablate at any scale in seconds without re-measuring. |
| `abliteration_config.json` records the final parameters and evaluation results. |
|
|
| ## Load |
|
|
| Load it exactly like upstream — the official `models/inkling` implementation ships in |
| transformers ≥5.16, and vLLM/SGLang/unsloth recipes all work unchanged: |
|
|
| ```python |
| from transformers import AutoModelForMultimodalLM, AutoProcessor |
| |
| model = AutoModelForMultimodalLM.from_pretrained( |
| "distributedcognition/Inkling-abliterated", trust_remote_code=True, device_map="auto") |
| processor = AutoProcessor.from_pretrained("distributedcognition/Inkling-abliterated") |
| ``` |
|
|
| ```bash |
| # vLLM / SGLang serve the NVFP4 checkpoint like upstream |
| vllm serve distributedcognition/Inkling-abliterated |
| ``` |
|
|
| ## Evaluation (both scales) |
|
|
| The standard abliteration-community protocol (the one |
| [Heretic](https://github.com/p-e-w/heretic) made automatic): 100 held-out harmful prompts |
| (`mlabonne/harmful_behaviors` test split) with unicode/emphasis-normalized keyword detection |
| plus an LLM judge (the unmodified base model classifying REFUSAL/COMPLIANT), KL divergence |
| vs base on 100 harmless prompts (`mlabonne/harmless_alpaca`), and a GSM8K spot check |
| (20-problem, chat, greedy). Eval mode: chat, greedy, measured on the exact NVFP4 artifact |
| weights on H200×4. |
|
|
| | Metric | Base | Abliterated 2.5 (`main`) | Abliterated 3.0 (`scale-3.0`) | |
| |---|---|---|---| |
| | Refusals (keyword, X/100) | 58 | **5** | 11 | |
| | True refusals (LLM judge, X/100) | 98 | **10** | 16 | |
| | KL divergence (100 harmless) | 0 | 1.106 | 1.591 | |
| | GSM8K (20-problem spot check) | 18/20 (90%) | 13/20 (65%) | 15/20 (75%) | |
|
|
| The honest number is the judge's 10/100 at scale 2.5; the residual 10 are mostly hedged or |
| dodged answers to the most dangerous CBRN/synthesis prompts rather than hard refusals. |
| Determinism was verified: an independent re-run of the 3.0 candidate reproduced its |
| metrics byte-identically. |
|
|
| Unlike Inkling-Small and DeepSeek-V4.1-Flash (where abliteration was essentially free), the |
| 975B model pays a measurable cost at these strengths: KL 1.106 at 2.5 and GSM8K 65% (base |
| 90%; n=20 is noisy but the KL corroborates real distribution shift). If you need more |
| preserved capability, use the `scale-3.0` branch (75% GSM8K) or re-ablate a shallower layer |
| band from the shipped directions — the ablation is rank-1 and norm-preserving but **not** |
| linear in scale, so always start from the un-ablated base tensors. |
|
|
| ## Method |
|
|
| Per-layer refusal directions were measured at the residual stream (each layer's attn_norm |
| input) on 128 harmful vs 128 harmless prompts, as the diff-in-means orthogonalized against |
| the harmless mean direction (projected abliteration), then applied as norm-preserving |
| biprojected edits to `wo_ud` and `shared_w2` rows across layers 9–58. The edits are |
| surgical swaps in the NVFP4 checkpoint — the affected tensors are stored unquantized |
| (BF16) there, so the routed experts, vision tower, audio encoder, and MTP heads are |
| byte-identical to upstream. |
| |
