README: two-img Tailwind dark classes (tracks HF site theme)

c053fe3 verified 8 days ago

5.48 kB

license: apache-2.0
base_model: thinkingmachines/Inkling
tags:
  - abliteration
  - uncensored
  - moe
  - multimodal
arxiv: 2406.11717

Inkling Abliterated (scale 2.5 on `main`, scale 3.0 on `scale-3.0`)

Responsible use

This is an uncensored research model. Its built-in refusals were removed permanently at the weight level, so it answers prompts the base model declines, including chemical and biological synthesis, cybercrime, weapons, harassment, and fraud. Use it for red-teaming, offensive-security research, and refusal-rate evaluation. It has no guardrails of its own: if you deploy it, add your own input and output moderation (for example Llama Guard).

What this is

Abliterated variant of thinkingmachines/Inkling (975B total / 41B active, multimodal MoE, 66 layers). The refusal direction was removed with norm-preserving biprojected abliteration (grimjim 2025), a weight-space refinement of directional ablation (Arditi et al. 2024), applied to the attention output projection (attn.wo_ud) and the shared-expert down projection (mlp.shared_experts.shared_w2) across layers 9–58, using per-layer directions.

Both evaluated scales ship as git branches (HF is git+LFS — switching branches is free; identical LFS objects are shared):

Item	Path / branch	Notes
Scale 2.5 (default)	`main`	Judge refusals 98 → 10/100, GSM8K 65%
Scale 3.0	`scale-3.0` branch	Judge refusals 98 → 16/100, GSM8K 75%

The repo mirrors the upstream NVFP4 checkpoint (same 33-shard layout, same tokenizer). Only the 100 abliterated tensors differ; everything else is byte-identical to upstream — the attention and shared-expert weights are stored unquantized (BF16) in NVFP4, so the 4-bit routed experts are untouched. refusal_directions.pt holds all 66 measured per-layer directions, so you can re-ablate at any scale in seconds without re-measuring. abliteration_config.json records the final parameters and evaluation results.

Load

Load it exactly like upstream — the official models/inkling implementation ships in transformers ≥5.16, and vLLM/SGLang/unsloth recipes all work unchanged:

from transformers import AutoModelForMultimodalLM, AutoProcessor

model = AutoModelForMultimodalLM.from_pretrained(
    "distributedcognition/Inkling-abliterated", trust_remote_code=True, device_map="auto")
processor = AutoProcessor.from_pretrained("distributedcognition/Inkling-abliterated")

# vLLM / SGLang serve the NVFP4 checkpoint like upstream
vllm serve distributedcognition/Inkling-abliterated

Evaluation (both scales)

The standard abliteration-community protocol (the one Heretic made automatic): 100 held-out harmful prompts (mlabonne/harmful_behaviors test split) with unicode/emphasis-normalized keyword detection plus an LLM judge (the unmodified base model classifying REFUSAL/COMPLIANT), KL divergence vs base on 100 harmless prompts (mlabonne/harmless_alpaca), and a GSM8K spot check (20-problem, chat, greedy). Eval mode: chat, greedy, measured on the exact NVFP4 artifact weights on H200×4.

Metric	Base	Abliterated 2.5 (`main`)	Abliterated 3.0 (`scale-3.0`)
Refusals (keyword, X/100)	58	5	11
True refusals (LLM judge, X/100)	98	10	16
KL divergence (100 harmless)	0	1.106	1.591
GSM8K (20-problem spot check)	18/20 (90%)	13/20 (65%)	15/20 (75%)

The honest number is the judge's 10/100 at scale 2.5; the residual 10 are mostly hedged or dodged answers to the most dangerous CBRN/synthesis prompts rather than hard refusals. Determinism was verified: an independent re-run of the 3.0 candidate reproduced its metrics byte-identically.

Unlike Inkling-Small and DeepSeek-V4.1-Flash (where abliteration was essentially free), the 975B model pays a measurable cost at these strengths: KL 1.106 at 2.5 and GSM8K 65% (base 90%; n=20 is noisy but the KL corroborates real distribution shift). If you need more preserved capability, use the scale-3.0 branch (75% GSM8K) or re-ablate a shallower layer band from the shipped directions — the ablation is rank-1 and norm-preserving but not linear in scale, so always start from the un-ablated base tensors.

Method

Per-layer refusal directions were measured at the residual stream (each layer's attn_norm input) on 128 harmful vs 128 harmless prompts, as the diff-in-means orthogonalized against the harmless mean direction (projected abliteration), then applied as norm-preserving biprojected edits to wo_ud and shared_w2 rows across layers 9–58. The edits are surgical swaps in the NVFP4 checkpoint — the affected tensors are stored unquantized (BF16) there, so the routed experts, vision tower, audio encoder, and MTP heads are byte-identical to upstream.

Inkling Abliterated (scale 2.5 on main, scale 3.0 on scale-3.0)