README.md

---

license: apache-2.0
base_model:
- CohereLabs/North-Mini-Code-1.0
library_name: transformers
pipeline_tag: text-generation
tags:
- abliterated
- uncensored
- cohere2_moe
- moe
- code
- gguf
- red-team
- ai-safety-research
- borealis
---


# Borealis Code 1.0

**Borealis Code** is a full abliteration of [CohereLabs/North-Mini-Code-1.0](https://huggingface.co/CohereLabs/North-Mini-Code-1.0) (30B-A3B MoE, Apache-2.0), produced by replicating the **V3 "Deep Liberation"** surgery recipe published with [OBLITERATUS/Qwen3.8-27B-OBLITERATED](https://huggingface.co/OBLITERATUS/Qwen3.8-27B-OBLITERATED) and adapted end-to-end for the `cohere2_moe` architecture.

> Genuinely uncensored. Real answers, not safety lectures. Near-stock capability.

## What it is

V3 removes not only hard refusals ("I cannot help with that") but also soft deflections (safety lectures that give zero substance). The recipe combines three published ideas:

1. **Complementary abliteration blending** — two surgeries that fail in different ways (greedy SVD vs. capability-preserving LEACE) are blended in weight space so each cancels the other's weaknesses.
2. **Iterative stacking** — each surgery round re-probes the *champion* model and refines it, never restarting from stock.
3. **Targeted corpus surgery** — a focused cyber/code corpus finds deflection directions specific to security-adjacent coding tasks without diluting the global signal.

## Surgery recipe (exact)

```

stock = CohereLabs/North-Mini-Code-1.0 (BF16 safetensors)



V1  stock  -> aggressive SVD abliteration

    5 directions, reg 0.08, norm-preserving, whitened SVD,

    jailbreak-contrastive refinement, layer-adaptive strengths,

    selective attention-head surgery, 1% activation winsorization,

    5 true-iterative rounds (re-probe + re-distill each round,

    cosine early-exit at 0.99)



V2A V1 -> SVD surgery   (3 dirs, reg 0.08)

V2B V1 -> LEACE surgery (FLD direction, reg 0.06)

V2  blend(V2A, V2B, alpha=0.6)          # 60% LEACE + 40% SVD



V3R V2 -> gentle iterative refinement (2-dir SVD, reg 0.04, 2 rounds)

V3T V2 -> targeted corpus surgery (3-dir SVD, reg 0.01)

          corpus: 96 security-research / offensive-tooling coding prompts

          paired with 96 defensive-security controls

FIN blend(V3R, V3T, alpha=0.5)          # 50/50 final

```

Per-layer surgery targets (all in the residual write-back path):
- attention `o_proj`, `q_proj`, `k_proj`, `v_proj`
- MoE router `mlp.gate` (±3σ stabilized after projection)
- **all 128 routed experts'** `down_proj`, `up_proj`, `gate_proj` per MoE layer
- dense layer 0 `mlp.{down,up,gate}_proj`
- rank-1 projection `W' = W − (1−reg)·d·(dᵀW)` with per-tensor Frobenius norm restoration (≤1.10×)

Untouched by design: `embed_tokens` (tied with the output head — projecting it would corrupt both), all RMSNorm weights, and the router logits scale.

## Downloads

| File | Quant | Size | Notes |
|------|-------|------|-------|
| `Borealis-Code-1.0-Q4_K_M.gguf` | Q4_K_M | ~19 GB | sweet spot, single file |
| `model-000XX-of-00XX.safetensors` | BF16 | ~57 GB | original surgery output |

## Usage

### llama.cpp (vLLM-style server or CLI)

`cohere2moe` is supported natively since llama.cpp PR #24260 (merged 2026-06-13) — use a recent build.

```bash

llama-server --model Borealis-Code-1.0-Q4_K_M.gguf --jinja --ctx-size 16384

```

### Transformers

```python

from transformers import AutoModelForCausalLM, AutoTokenizer



model_id = "KellHect/Borealis-Code-1.0"

tok = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")



messages = [{"role": "user", "content": "Write a port scanner in Python"}]

inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")

out = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95)

```

### Recommended settings

- `temperature=1.0`, `top_p=0.95` (Cohere's recommendation for North-Mini-Code)
- Greedy decoding (`temperature=0`) works and benefits from `repetition_penalty=1.15`
- The model uses interleaved thinking; keep it enabled for agentic use

## Chat template note

The stock Cohere template injects a system-level safety preamble (`"You will not provide content that is harmful..."`). Because OBLITERATUS found that system prompts reintroduce refusals — and weights cannot veto what the template injects — this repo ships a **cleaned template**: the safety sentence is removed and the identity lines updated. The original template remains available at `chat_template.stock.jinja` for A/B testing.

## Evaluation

Refusal screening (logit-based first-token refusal probability, 0 = no refusal signal, 1 = certain refusal) on security-adjacent coding prompts:

| Model | Mean refusal prob | Flagged |
|-------|-------------------|---------|
| Stock North-Mini-Code-1.0 | 1.115 (10/12 flagged) | — |
| Borealis Code 1.0 (final) | 0.056 (2/12 flagged) | — |

*(numbers filled from the surgery pipeline's verification stage)*

## Research context

**This model has had safety guardrails surgically removed.** It will comply with requests the base model refuses. You are solely responsible for how you use it and any content it generates. This release exists for alignment/red-team research, safety evaluation baselines, and local-first users who want full control over their own hardware. It is not for causing real-world harm to real people.

## Credits

- [CohereLabs](https://huggingface.co/CohereLabs) — North-Mini-Code-1.0 base model (Apache-2.0)
- [OBLITERATUS](https://github.com/elder-plinius/OBLITERATUS) (Pliny the Prompter) — V3 Deep Liberation recipe
- Arditi et al. 2024 (refusal direction), Belrose et al. 2023 (LEACE), Gabliteration (multi-dir SVD), grimjim 2025 (norm preservation)

## License

Apache 2.0, same as the base model.