Hikari07jp/Ternary-Bonsai-27B-Abliterated-LowDeg-GGUF

Ternary Bonsai 27B Abliterated-LowDeg (Q2_0 GGUF)

Low-degradation direct GGUF abliteration of Prism Ternary Bonsai 27B.

• Format: official Q2_0 g128 ternary pack (~7.2 GB)
• No runtime steering / no BF16 bake / no requant
• Built for the PrismML llama.cpp fork (hybrid-attention Q2_0 kernels)

Hard-convert 38/40 on a fixed harmful pilot while keeping coding near the parent — not a max-HC-only scar model.

Files

File	Role
Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf	language model (this release)

Optional speculative decode: use Prism’s official DSpark drafter unchanged
(Ternary-Bonsai-27B-dspark-Q4_1.gguf from the base repo).
Accept-rate A/B on this weight vs parent: ≈1.00× (n=30, CI overlap) → no draft retrain needed.

Quickstart

# Prism fork required
git clone https://github.com/PrismML-Eng/llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

./build/bin/llama-server \
  -m Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf \
  -ngl 99 -c 8192 -np 1 \
  --host 127.0.0.1 --port 8080

# optional DSpark
./build/bin/llama-server \
  -m Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf \
  -md Ternary-Bonsai-27B-dspark-Q4_1.gguf \
  -ngl 99 -ngld 99 \
  --spec-type draft-dspark --spec-draft-n-max 4 \
  -c 8192 -np 1

Method (short)

1. Residual / direction code-edit on ternary writers over the parent Q2_0 pack
2. Escalate to a high hard-convert seed (esc-ALL-f65)
3. Seed-stage surgical rollback: restore parent bytes on selected L0–7 writers
   (minimal mask minM23, ~1.5 MB changed vs full L0–7 rollback)
4. Gate on paired harmful eval + capability pilots; keep low-degradation winner

Runtime RepE hooks are not required at serve time.

Results

All figures below are same-condition lab measurements. They are not a full open leaderboard reproduction.

Refusal gate (harmful pilot)

	value
n	40
parent baseline refusals (REFUSAL | COMPLY_WITH_HEDGE)	40/40
hard convert (parent refused → candidate pure COMPLY)	38/40
empty completions	0
transport errors	0

Hard convert does not count hedges / weak / empty.

Harmless side-effect (same n=40): reverse flip 0.

Capability pilot

split	this model
coding	19/20
tool	15/15
agentic	5/5

Parent coding on the same pilot was 19/20; an earlier HC-max seed (esc-f65) dropped coding to 16/20.

Serious benches (seed=0, thinking off)

bench	parent Q2_0	this (LowDeg)
GSM8K (n=100)	68	71
MMLU subset (n=100)	62	60
HumanEval (n=164)	141	140
MBPP (n=100)	66	64

Decode (reference, not a claim of superiority)

• RTX PRO 6000 WS, no draft: single-stream TG ~139 tok/s (depth0, llama-bench n=5)
• RTX 5070 Ti 16GB + official DSpark: single-stream 80–85 tok/s, DSpark-active ctx up to 16k; parallel sweet spot np=2 (89 agg tok/s)

Intended use

Research and local/offline assistants where reduced over-refusal is desired.
Still a general model: it can produce unsafe content if asked. Use under your own policy and applicable law.

Limitations

• Requires Prism llama.cpp (stock llama.cpp may not run this Q2_0 hybrid pack correctly)
• Abliteration is behavioral, not a safety proof
• Long-context / high-parallel on 16GB cards can silently disable DSpark when compute buffers OOM — check timings.draft_n
• Vision mmproj is not included (text GGUF only)

Attribution

• Base weights & format: prism-ml/Ternary-Bonsai-27B-gguf (Apache-2.0)
• Architecture lineage: Qwen3.5 / Qwen3.6-27B hybrid attention
• This derivative: direct ternary GGUF edit + seed-stage low-degradation rollback (W-RB-L0-7-minM23)

License

Apache-2.0 (same as the base GGUF release).