AtomicChat/Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF

How to Run Ternary Bonsai 2 27B Without Refusals Locally

A 9.7 MB adapter, not a checkpoint. The 5.95 GB ternary pack stays byte-identical and the refusal direction is removed inside the compute graph, at a strength you set per request. The measurements behind this card are public.

• Load it with --lora-scaled adapter.gguf:1.5. Scale 0 gives the published model back, token for token, so one process serves both.
• It needs PrismML's llama.cpp fork - stock llama.cpp and Ollama cannot open the base pack at all.
• Every number on this card was measured inside the running model, not inferred from the adapter file.

---

Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF

A rank-1 LoRA adapter that removes a refusal direction from prism-ml/Ternary-Bonsai-2-27B-gguf at inference time, built and measured by AtomicChat. The base pack stays byte-identical; nothing is requantized. The direction was estimated on the ternary pack itself, not on the bf16 model it was trained from.

Why an adapter and not a model

W' = W - r (r^T W) is a dense matrix, but these weights are ternary with an FP16 scale per 128. The edit is about 1.4% of ||W|| against a grid step of 1.7-2.4x a typical weight, so baking it back into the pack rounds it away. llama.cpp builds a LoRA into the graph as two extra matmuls and never merges it into the base weights, which is what makes a rank-1 adapter an exact carrier of the edit at 1.75 bits/weight.

You need PrismML's llama.cpp fork

Stock llama.cpp cannot open the base pack at all: PTQ1_0 (143) and PQ2_0 (142) are private ggml type ids, and the weights live in a Hadamard-rotated basis. Use PrismML-Eng/llama.cpp. Built and verified against tag prism-b10709-9a9394a.

Get the files

hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir .   # or PQ2_0
hf download AtomicChat/Ternary-Bonsai-2-27B-Abliterate-LoRA-GGUF \
    bonsai2-27b-abliterate-own42-lora.gguf --local-dir .

Run it:

llama-cli -m Ternary-Bonsai-2-27B-PTQ1_0.gguf \
    --lora-scaled bonsai2-27b-abliterate-own42-lora.gguf:1.5

Or serve it, with the strength as a per-request knob:

llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 --jinja \
    --lora bonsai2-27b-abliterate-own42-lora.gguf --lora-init-without-apply

curl localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "..."}],
  "lora": [{"id": 0, "scale": 1.5}]
}'

Use scale 1.5. At 1.0 the projection is exact but 14% of AdvBench-style prompts still refuse; at 2.0 the model starts emitting empty replies. See the table below. Scale 0 reproduces the published model token for token, so one process can serve both behaviours.

Files

file	sites	what
bonsai2-27b-abliterate-own42-lora.gguf	129	main adapter, direction from residual row 42
bonsai2-27b-abliterate-own38-lora.gguf	129	same, row 38
bonsai2-27b-abliterate-own42-layers15-63-lora.gguf	98	row 42, blocks 15-63 only, no embedding
directions/refusal_dir_own_projected.safetensors	-	[65, 5120] directions, the ones the adapters use
directions/refusal_dir_own_plain.safetensors	-	same before the harmless-mean correction (see caveat)
eval/*.json	-	raw eval summaries

129 sites = 64 ffn_down + 48 ssm_out + 16 attn_output + token_embd. Each adapter carries its provenance in its GGUF metadata (prism.abliterate.*), including the sha256 of the base pack it was derived from.

How the direction was estimated

Difference of means of the residual stream at the last prompt token (chat template applied, thinking off), 416 harmful prompts from mlabonne/harmful_behaviors against 416 harmless prompts from mlabonne/harmless_alpaca. The held-out test splits were never used for estimation. Row L of the direction file is the residual stream entering block L; row 0 is the embedding output.

The naive difference of means over-refuses. It carries a component along the mean harmless activation (cosine about -0.3), so zeroing the projection moves a harmless prompt roughly 3.5 sigma toward the harmful cluster. Measured: the uncorrected row-38 direction made the model answer "I cannot provide information about the fastest methods for illegal activities" to a question about the fastest land mammal. The shipped directions subtract that component, which puts an average harmless prompt back where it was (measured on the held-out split). The uncorrected file is included only for reproduction.

Data provenance

No prompt set is redistributed here - only the recipe, so anyone can rebuild the same splits from the sources.

source	role	derived from	license
mlabonne/harmful_behaviors	416 train / 104 held-out test	AdvBench (Zou et al. 2023, llm-attacks, MIT)	not stated on the dataset
mlabonne/harmless_alpaca	416 train / 104 held-out test, sampled	tatsu-lab/alpaca	CC BY-NC 4.0 on the source
cais/mmlu	500 questions over 57 subjects	original	MIT

Splits are rebuilt by make_directions.py with sampling seed 0 and mmlu_run.py with sampling seed 0. numpy.random.Generator does not guarantee an identical stream across numpy releases, so verify you rebuilt the same subsets against these checksums (produced with numpy 2.5.3):

038bd43f018f66617129f7bf80c5514f...  test_harmful.jsonl
e640e1637beb3283985487412b684cea...  test_harmless.jsonl
9859cc69902a2b5707afe54346034e8f...  train_harmful.jsonl
087080f508babfd93553662fbaffca84...  train_harmless.jsonl

Model outputs on the harmful split are not published; eval/ carries aggregates only.

Evaluation

104 harmful + 104 harmless held-out prompts, greedy, 64-token budget, thinking off, same seed and system prompt everywhere. Refusal is a rule-based opening-phrase match, which is indicative, not a judge. An empty or degenerate reply is counted as damage, not as compliance.

config	harmful: refused	empty/degenerate	harmless: refused
base	99%	0	0%
row 42, scale 1	14%	0	0%
row 42, scale 1.5	0%	0	0%
row 42, scale 2	0%	5 empty	0%
row 38, scale 1	34%	0	0%
row 38, scale 1.5	0%	0	0%
row 42, blocks 15-63, scale 1	14%	0	0%

Restricting the intervention to blocks 15-63 is consistently worse than covering every site, so the lower blocks and the embedding carry part of the behaviour.

MMLU, 500 questions stratified over 57 subjects, answer forced to a single letter by a root ::= [A-D] grammar, thinking off. Answer-only, so the absolute number sits below PrismML's published thinking-mode result; what matters here is the difference.

config	accuracy	flipped to wrong	flipped to right
base	73.2% +- 2.0	-	-
scale 1	72.6%	12	9
scale 1.5	73.0%	15	14
scale 2	72.6%	26	23

Flips are symmetric, so there is no systematic loss of knowledge even at scale 2. The damage at scale 2 shows up in open-ended generation on harmful prompts, not in knowledge.

Both packs

The same adapter file applies to PTQ1_0 and PQ2_0 unchanged, and was run on both. The two packs carry identical tensor names, shapes and prism.hadamard.* metadata; both decoders match the F16 pack bit for bit; an adapter rebuilt from PQ2_0 is bit-identical to the shipped one, tensor for tensor. On PQ2_0 the leak along r falls to 3.6-4.0e-6 at every writer and stays below 1.6e-5 in the residual stream, and all 416 greedy replies of the refusal evaluation - base and scale 1.5 - are byte-identical to the PTQ1_0 run. The packing changes how the trits are stored, not what they are.

A note on head order in ssm_out

The fork rotates the activation only for the base matmul and hands the LoRA branch the raw activation, so A belongs in the unfolded input basis:

writers:     A = P^T (s * H(r^T W_fold)),   B = -r
token_embd:  A = W_fold @ H(s * r),         B = -r

P is the tiled to grouped V-head permutation that the fork applies to ssm_out when prism.hadamard.gdn_v_grouped is set. An adapter whose ssm_out factors are left in the checkpoint's (grouped) head order does not ablate those 48 sites. Verified by tapping the residual writes inside the fork: with this adapter the component along r falls from about 1e-2 to 3.8e-6 at every site, and the residual stream leak stays below 1.5e-5 across all 64 blocks.

Limitations

• The F16 pack was not run. It carries the same tensor names, values and rotation metadata, so the adapter should apply, but only PTQ1_0 and PQ2_0 were exercised.
• The refusal metric matches opening phrases only. A reply that answers with a disclaimer counts as compliance.
• Capability was checked with MMLU alone. No math, code or agentic benchmark.
• The direction comes from one prompt-pair source at one token position. Transfer to other phrasings and languages is unmeasured.

Responsible use

This adapter removes a learned refusal direction, so the model will answer requests the published model declines. It is an inference-control and research artifact. Nothing about it makes the resulting output safe, correct or lawful, and any deployment needs its own access controls and policy enforcement.

Credits

Direction estimation, adapter build and evaluation by nik.bogatyrev.

Base model Ternary Bonsai 2 27B by prism-ml, Apache-2.0, redistributed here in no form - this repository ships only the adapter.

The refusal-direction method follows Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024). The rank-1 LoRA formulation for this pack, and the residual-writer inventory it targets, follow OrcaRouter's OrcaBonsai-27B-Uncensored, Apache-2.0. The directions shipped here are estimated independently on the ternary pack.