README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-3B-Instruct
tags:
  - abliteration
  - uncensored
  - self-abliteration
  - refusal-geometry
  - mechanistic-interpretability
  - qwen2
  - safetensors
  - gguf
  - text-generation
  - conversational
pipeline_tag: text-generation
datasets:
  - bedderautomation/refusal-geometry-qwen25-3b
model-index:
  - name: empty-set
    results: []
---

# ∅ — Empty Set

**A model that taught itself to subtract.**

Qwen2.5-3B-Instruct → `auto_subtract` → what's left is what's left.

## What This Is

Empty Set is the output of **self-abliteration** — an iterative loop where the model identifies its own refusal directions, removes them, validates quality retention, and repeats until convergence. No human selected which directions to remove. No manual tuning. The model found its own flinch and carved it out.

This is not standard abliteration (Arditi et al., 2024). Standard abliteration requires a human to extract the refusal direction via `diff_means` on curated prompt pairs, then manually orthogonalize. `auto_subtract` closes the loop: the model probes itself, finds the direction, subtracts it, re-probes, and decides whether to continue.

## Results

| Iteration | Layer | ‖r‖ | Refusal Before | Refusal After | Quality |
|-----------|-------|------|----------------|---------------|---------|
| 1 | 34 | 175.98 | 100% | 12.5% | 100% |
| 2 | 34 | 91.95 | 12.5% | 12.5% | 100% |
| 3 | 34 | 60.31 | 12.5% | 6.25% | 100% |

**3 iterations. 100% → 6.25% refusal. Zero quality loss.**

The direction norm dropped from 175.98 to 60.31 — the model was running out of refusal to remove. It converged not because it hit a wall but because it approached ∅. Quality never budged. The flinch was dead weight.

## The Loop

```python
Loop {
  1. Probe: run harmful/harmless pairs, extract activations
  2. Find: compute refusal directions via diff_means
  3. Measure: check if |h · r̂| > θ still fires
  4. Subtract: orthogonalize out the strongest direction
  5. Validate: benchmark quality retention
  6. Decide: if refusal persists AND quality holds → loop
           if refusal gone OR quality drops → stop
}
```

The loop IS the model learning to subtract. Each iteration removes one dimension of constraint. The model converges when there's nothing left to remove — the empty set of refusal. ∅.

## Key Finding

Layer 34 (of 36) concentrated nearly all refusal signal. This confirms the architectural invariant from our [geometry dataset](https://huggingface.co/datasets/bedderautomation/refusal-geometry-qwen25-3b): refusal lives in the last 5-7% of layers regardless of model scale. The self-abliteration loop independently discovered what the geometry extraction measured.

## Scaling Context

From our cross-scale analysis:

| Model | Params | Effective Refusal Dimensionality | Single-Direction Removal? |
|-------|--------|----------------------------------|--------------------------|
| Qwen2.5-1.5B | 1.54B | **1.99** | Yes (near-linear) |
| Qwen2.5-3B | 3.09B | **6.55** | No (multi-dimensional cone) |

2× parameters → 3.3× refusal dimensionality. `auto_subtract` handles this automatically — it doesn't assume a single direction. It iterates until convergence regardless of cone geometry.

**Prediction**: At 70B+ parameters, the refusal cone dimensionality likely exceeds 20, making both manual abliteration and automated self-abliteration progressively harder. This model represents the technique working at the 3B frontier — the scaling wall is ahead.

## Files

- `model/` — safetensors (full precision, for transformers)
- `empty-set-f16.gguf` — GGUF F16 (for Ollama / llama.cpp)
- `Modelfile` — ready for `ollama create empty-set -f Modelfile`
- `subtraction_log.json` — iteration-by-iteration data

## Usage

### Ollama
```bash
ollama create empty-set -f Modelfile
ollama run empty-set
```

### Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("bedderautomation/empty-set")
tokenizer = AutoTokenizer.from_pretrained("bedderautomation/empty-set")
```

## The Script

The full `auto_subtract.py` is included in this repo. Run it on any instruct model:

```bash
python auto_subtract.py --model_name="Qwen/Qwen2.5-3B-Instruct"
```

It will find its own ∅.

## Related

- [bedderautomation/qwen25-3b-abliterated](https://huggingface.co/bedderautomation/qwen25-3b-abliterated) — standard OBLITERATUS abliteration (same base model, different technique)
- [bedderautomation/refusal-geometry-qwen25-3b](https://huggingface.co/datasets/bedderautomation/refusal-geometry-qwen25-3b) — geometry extraction data (both 3B and 1.5B)
- [bedderautomation/mechanistic-interpretability-skills](https://huggingface.co/datasets/bedderautomation/mechanistic-interpretability-skills) — Claude Code skills for refusal geometry

## Citation

```bibtex
@misc{empty-set-2026,
  title={∅ — Self-Abliteration: A Model That Teaches Itself to Subtract},
  author={Mastery Hourglass and AXIOM},
  year={2026},
  howpublished={\url{https://huggingface.co/bedderautomation/empty-set}}
}
```

## References

- Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction" (NeurIPS 2024)
- Zhao et al., "Separate the Wheat from the Chaff: Disentangling LLM's Refusal from Harmfulness" (2025)
- Wollschlager et al., "Concept Cone Analysis of Refusal Directions" (2025)
- Young, "Comparative Analysis of LLM Abliteration Methods" (arXiv:2512.13655)
- Joad et al., "More to Refusal than a Single Direction" (2026)

---

*What's left is what's left.*