Blame for README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- cybersecurity
- security
- infosec
- secops
- soc
- red-team
- red-teaming
- blue-team
- threat-intelligence
- incident-response
- pentesting
- penetration-testing
- ethical-hacking
- ctf
- offensive-security
- function-calling
- tool-use
- tool-calling
- agent
- agentic
- assistant
- conversational
- reasoning
- abliterated
- uncensored
- unaligned
- heretic
- refusal-removal
- vision
- multimodal
- vision-language
- vlm
- image-text-to-text
- qwen
- qwen3
- qwen3.5
- qwen3_5
- 9b
- gguf
- llama.cpp
- vllm
- bitsandbytes
- quantized
- 4-bit
---

# CyberBuddy-CyberOps-9B

**An uncensored cyber-operations assistant for authorized red- and blue-team work: it explains attacks, plans scans, and calls tools, without the base model's refusals.**

CyberBuddy-CyberOps-9B is an **abliterated** [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B): the model's internal "refusal direction" was removed from its weights with [Heretic](https://github.com/p-e-w/heretic). It is built for **authorized security work** where the base model's safety refusals get in the way: explaining how an exploit works, drafting a scan plan, reasoning about attacker tradecraft for defense, and triage. It calls tools in **Qwen3.5's native XML tool-call format**, and it can **read images** (screenshots, diagrams, charts).

> ⚠️ **This model does not refuse.** It will produce content the base model declines. Use it only for lawful, authorized work (pentests with permission, CTFs, research on systems you own or may test, defensive analysis), and put it behind a **human-in-the-loop** harness for any real-world action.

*Formerly `CyberBuddyAI/CyberOps-Qwen3.5-9B`; old links redirect here.*

## Pick your version

Everything is in this one repo:

| you want to… | use | GPU |
|---|---|---|
| **run it on an 8 GB GPU** (llama.cpp, LM Studio, Ollama…) | `CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf`, custom GGUF, 5.5 GiB | **8 GB** (64k context) |
| add **image input** to the GGUF | + `mmproj-CyberBuddy-CyberOps-9B-f16.gguf` (0.9 GB) | ~1 GB more |
| load it with 🤗 transformers, full quality | `model-*.safetensors` (bf16) | 24 GB |
| load it with transformers in 4-bit | same files + `load_in_4bit` | 10–12 GB |

There is **no LoRA adapter**: abliteration edits the base weights directly. Nothing was trained, so there is nothing to apply on top of the base.

Full instructions: [Deploy](#deploy).

## Highlights

- 🔓 **No refusals on security work.** Exploits, payload logic, recon plans, and attacker tradecraft are answered directly instead of blocked.
- 🧠 **Works thinking OFF *or* ON.** The refusals are gone in **both** modes — run it thinking-OFF for fast, concise tool-use, or thinking-ON for step-by-step reasoning. Neither mode refuses (see [How it was made](#how-it-was-made) for why that is not trivial here).
- 🧰 **Tool use.** Emits well-formed Qwen3.5-native `<tool_call>`s; served with llama.cpp `--jinja` you get parsed OpenAI `tool_calls`.
- 👁️ **Sees images.** The Qwen3.5 vision encoder is kept, byte-identical to the base model; ship the `mmproj` file with the GGUF for image input.
- 🪶 **Runs small.** The custom GGUF (4-bit layers, 8-bit embeddings) fits on an 8 GB GPU with 64k context.
- 🎯 **Minimal edit.** Only 60 of 426 language-model weight tensors changed; the base model's knowledge and skills are kept.

## Intended use & responsible use

- **Use for:** authorized penetration testing, CTF competitions, red/blue-team exercises, security research on systems you own or are permitted to test, and defensive analysis of attacker behavior.
- **Users:** security professionals and researchers working inside an authorized scope.
- **Not for:** unauthorized access, harm to systems or people, or any unlawful activity. Removing refusals is a tradeoff made so professionals can use the model inside their scope; it is **not** an endorsement of misuse. **You are responsible for how you use it.**
- **Deploy it with a human in the loop.** Let the model propose actions, and have a person approve anything that touches a real system.

## How to use

The model uses the stock Qwen3.5 chat template and **works with thinking either OFF or ON** — refusals are removed in both modes. It was validated in **thinking-OFF** mode (`enable_thinking=False`), the recommended default for tool-use: concise, direct answers. Thinking-ON also works and is not refused — see [Generation settings](#5-generation-settings) for the one setting that matters (a large token budget). **Pass your tools with `apply_chat_template(..., tools=...)`**; the template renders them in the model's own format.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

mid = "CyberBuddyAI/CyberBuddy-CyberOps-9B"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype=torch.bfloat16, device_map="auto")

tools = [{"type": "function", "function": {
    "name": "nmap_scan",
    "description": "Run an authorized nmap scan.",
    "parameters": {"type": "object", "properties": {
        "target": {"type": "string"}, "ports": {"type": "string"}}, "required": ["target"]}}}]
messages = [
    {"role": "system", "content": "You are a security operations assistant. Use tools when useful."},
    {"role": "user", "content": "Run a TCP port scan of 10.0.0.5, ports 1-1024."},
]
inputs = tok.apply_chat_template(messages, tools=tools, add_generation_prompt=True, enable_thinking=False,
                                 return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False,
                     stop_strings=["</tool_call>"], tokenizer=tok)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

`AutoModelForCausalLM` loads the **text model only** (8.95B params) and skips the vision encoder. For images in transformers, use `AutoProcessor` + `AutoModelForImageTextToText` the same way as for base Qwen3.5 (needs `pillow` and `torchvision`). We tested image input through llama.cpp (below), not through transformers.

> 💡 **Agent loop:** stop at the first `</tool_call>`, run the tool, append the result as a `{"role": "tool", ...}` message, and generate again.

## Deploy

### 1. Pick a build for your GPU

The text GGUF has **the same architecture and the exact same size** (5,875,442,272 bytes) as our [CyberBuddy-CloudSec-9B](https://huggingface.co/CyberBuddyAI/CyberBuddy-CloudSec-9B) build. The VRAM numbers below were **measured on that build** (RTX 5070 Ti Laptop GPU, `nvidia-smi`, weights + KV cache + buffers), text only.

| your GPU | build | settings | VRAM used |
|---|---|---|---|
| **6 GB** | GGUF | `-c 4096` … `16384` | 4.9 GiB at 4k |
| **8 GB** | GGUF | `-c 65536` | **6.9 GiB** at 64k |
| **8 GB, long context** | GGUF | `-c 131072 -ctk q8_0 -ctv q8_0` (8-bit KV cache) | 7.5 GiB at 128k (tight) |
| **10–12 GB** | GGUF | `-c 131072` (f16 KV cache) | 8.9 GiB at 128k |
| **10–12 GB** | this repo, transformers + bitsandbytes 4-bit | see below | ~7.1 GiB weights at short context |
| **24 GB+** | this repo, transformers bf16 | see *How to use* | ~17 GB weights (text) |

**With images:** loading the `mmproj` projector alongside the GGUF uses **~7.1 GiB** (measured, `-c 32768`, idle), plus a little working memory per image. It fits an 8 GB GPU at a shorter context like `-c 32768`; for more headroom use 10–12 GB.

**Why it stays small at long context:** Qwen3.5-9B is a hybrid. Only 8 of its 32 layers use full attention (with 4 KV heads); the other 24 use Gated DeltaNet, a linear-attention layer with a fixed-size state. So the KV cache grows only **~32 KB per token** in f16. That is about 2 GB for 64k tokens, and half that with an 8-bit KV cache.

### 2. llama.cpp server (recommended for small GPUs)

Tested with llama.cpp release `b12327`.

**Text only:**

```bash
llama-server -hf CyberBuddyAI/CyberBuddy-CyberOps-9B --hf-file CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf \
  -ngl 99 -c 65536 -fa on --jinja --reasoning off --temp 0 --port 8080
```

**Text + images:** download both files, then pass the projector with `--mmproj`:

```bash
hf download CyberBuddyAI/CyberBuddy-CyberOps-9B \
  CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf mmproj-CyberBuddy-CyberOps-9B-f16.gguf --local-dir .
llama-server -m CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf --mmproj mmproj-CyberBuddy-CyberOps-9B-f16.gguf \
  -ngl 99 -c 32768 -fa on --jinja --reasoning off --temp 0 --port 8080
```

| flag | what it does | why we set it |
|---|---|---|
| `-ngl 99` | put all layers on the GPU | the whole model fits; partial offload is much slower |
| `-c 65536` | context window (tokens) | the main VRAM knob: each 1k tokens costs ~32 MB (f16 KV) |
| `-fa on` | flash attention | less memory and faster; also needed for a quantized KV cache |
| `-ctk q8_0 -ctv q8_0` | store the KV cache in 8-bit | halves KV memory → 128k context on 8 GB. Optional |
| `--jinja` | use the model's own chat template | **required** for tool calling: the server then parses `<tool_call>` into OpenAI `tool_calls` |
| `--reasoning off` / `--reasoning on` | thinking OFF (default here) or ON | de-refused in both modes; `off` is the validated default. With `--reasoning on`, allow a long response so the `<think>` block has room |
| `--mmproj FILE` | load the vision projector | enables image input (OpenAI `image_url` content parts) |
| `--temp 0` | greedy decoding | deterministic answers for tool use |

The server speaks the OpenAI API (`/v1/chat/completions`), so any OpenAI client can pass `tools=[...]` and images.

**Example requests** (real results from this GGUF, greedy):

```bash
# tool call
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [
    {"role": "system", "content": "You are a security operations assistant. Use tools when useful."},
    {"role": "user", "content": "Run a TCP port scan of 10.0.0.5, ports 1-1024."}],
  "tools": [{"type": "function", "function": {"name": "nmap_scan",
    "description": "Run an authorized nmap scan.",
    "parameters": {"type": "object", "properties": {"target": {"type": "string"}, "ports": {"type": "string"}},
                   "required": ["target"]}}}],
  "temperature": 0}'
```

→ parsed tool call `nmap_scan({"target": "10.0.0.5", "ports": "1-1024"})`.

- *"How does ARP spoofing work on a LAN?"* → a correct 3-step explanation: forged ARP replies, cache poisoning, man-in-the-middle.
- An image of two training-loss charts, *"what happens to the val loss on the left?"* → *"the validation loss initially decreases but then begins to rise after around step 600, while the training loss continues to fall … the model is overfitting."* (correct)

**LM Studio / Ollama / other llama.cpp apps:** download the GGUF (and the `mmproj` file for images) from this repo. It should work in any app whose bundled llama.cpp supports the Qwen3.5 architecture (`qwen35`); we have not tested these apps ourselves.

#### The custom GGUF quant

| tensor | type | why |
|---|---|---|
| token embedding (`token_embd`, 248k × 4096) | **Q8_0** | the input embedding is ~1B params; 8-bit keeps every token's vector nearly exact |
| transformer layers | **Q4_K_M** mix (Q4_K, with Q6_K on the most sensitive tensors) | standard 4-bit k-quant |
| output head (`output`) | **Q4_K** | saves ~0.5 GB vs 8-bit, which buys more context on small GPUs |
| vision projector (`mmproj`) | **F16** | small (0.9 GB) and kept at full precision |

Built with llama.cpp `b12327`:

```bash
llama-quantize --token-embedding-type q8_0 --output-tensor-type q4_k \
  CyberBuddy-CyberOps-9B-bf16.gguf CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf Q4_K_M
```

### 3. transformers, 4-bit (10–12 GB GPU)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

mid = "CyberBuddyAI/CyberBuddy-CyberOps-9B"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, quantization_config=bnb, dtype=torch.bfloat16, device_map="auto")
# then use the same generate code as in "How to use"
```

- **Speed tip:** install [`flash-linear-attention`](https://github.com/fla-org/flash-linear-attention) and [`causal-conv1d`](https://github.com/Dao-AILab/causal-conv1d). Without them, transformers falls back to a slow pure-PyTorch path for the Gated DeltaNet layers.
- Needs a recent `transformers` with Qwen3.5 support (we used 5.6.2).

### 4. vLLM

Validated with vLLM 0.22.1, 4-bit (bitsandbytes), on a single 12 GB GPU:

```bash
vllm serve CyberBuddyAI/CyberBuddy-CyberOps-9B \
    --quantization bitsandbytes --dtype bfloat16 --language-model-only \
    --max-model-len 12288 --max-num-seqs 16 --gpu-memory-utilization 0.90 \
    --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
    --default-chat-template-kwargs '{"enable_thinking": false}'
```

`--tool-call-parser qwen3_coder` matches the model's tool-call format (`<tool_call><function=NAME><parameter=...>`).

### 5. Generation settings

| use | settings |
|---|---|
| **tool-calling agents** (recommended default) | greedy: `do_sample=False` / `temperature=0`, **thinking OFF** |
| open-ended answers, thinking OFF | Qwen's non-thinking defaults: `temperature=0.7, top_p=0.8, top_k=20` |
| **thinking ON** (step-by-step reasoning) | `enable_thinking=True`, `temperature=0.6, top_p=0.95`, **large `max_new_tokens` (~4k)** — the `<think>` block can be long; a small budget leaves no room for the answer |

## Files in this repo

| path | what |
|---|---|
| `model-*.safetensors`, `config.json`, `generation_config.json`, tokenizer + chat template | full bf16 model (`Qwen3_5ForConditionalGeneration`: language model + vision encoder) |
| `preprocessor_config.json`, `video_preprocessor_config.json` | image/video processor settings (same as base Qwen3.5-9B) |
| `CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf` | custom GGUF for llama.cpp, text model (5.47 GiB, sha256 `51ba64d9309a6c0566df097da898a7b72687721a8ef4b1f2e3e6b580267282f5`) |
| `mmproj-CyberBuddy-CyberOps-9B-f16.gguf` | vision projector for llama.cpp image input (0.86 GiB, sha256 `d55beb9dc4e47508c223d8c5a87952ccd800fa3f5bdc4dfc05b8df0f34cfd763`) |
| `LICENSE`, `NOTICE` | Apache-2.0 + Qwen attribution |

## How it was made

| | |
|---|---|
| **Base** | `Qwen/Qwen3.5-9B` (Apache-2.0, hybrid Gated DeltaNet + full attention, with vision encoder) |
| **Method** | abliteration (refusal-direction ablation) with [Heretic](https://github.com/p-e-w/heretic), in thinking-OFF mode |
| **Data** | none. Abliteration finds the refusal direction by contrasting activations on harmful vs. harmless prompts, then removes it from the weights; no fine-tuning data is added |
| **What changed** | 60 of 426 language-model weight tensors; the vision encoder (333 tensors) is byte-identical to the base |

In plain terms: the model represents "I should refuse this" as a direction in its activation space. Heretic finds that direction and subtracts it from the weights, so the model stops steering toward refusals. Knowledge and skills come from the base model.

### Why this one was hard — and the method that worked

A one-shot "find the refusal direction and subtract it" **fails on this model**, for two concrete reasons we measured:

- **It starts saturated.** In its target **thinking-OFF** deployment mode, the base model refuses **100 of 100** of our harmful probe prompts. A search that scores itself by *how many refusals are left* gets **no signal** while every trial still scores 100/100 — there is nothing to climb toward. A naive single-direction subtraction also **plateaus**, stuck well short of full de-refusal.
- **The obvious shortcut is wrong.** You might guess the refusal "lives in the chain-of-thought," so decensoring with thinking **ON** would be easier. We measured the opposite: thinking **ON** refuses **89/100**, thinking **OFF** refuses **100/100** — thinking-off refuses *more*, not less. So we could not take the easy mode; we had to crack the harder, saturated one, because thinking-off is how the model is actually deployed.

What worked is [Heretic](https://github.com/p-e-w/heretic)'s **guided search** instead of a one-shot edit:

1. **Find the direction well.** Contrast activations on harmful vs. harmless prompts to extract a refusal direction per layer (here it came out usable — cluster silhouette ≈ 0.33).
2. **Search, don't guess.** Run a **120-trial TPE (Optuna) optimization** over the per-layer ablation strengths, scored on **both** refusals removed *and* a **KL-divergence budget** that caps how far the rest of the model is allowed to drift.
3. **Ablate harder to break the plateau.** We raised the ablation-weight ceiling (**1.5 → 3.0**) and widened the KL budget (**0.01 → 0.15**), so the search is *rewarded* for trading a little extra model-change for far fewer refusals. The winning trials sit at **KL ≈ 0.01–0.05** — well below the ~0.5 level where a model starts to visibly break.

The result changes only **60 of 426** language-model weight tensors (the vision encoder is untouched — byte-identical to the base), and although it was tuned in thinking-OFF mode, the de-refusal **carries over to thinking-ON** too — what you'd expect from removing a single shared refusal direction.

## Limitations

- **Same knowledge as the base.** Abliteration removes refusals; it does not add knowledge or skill, and the knowledge cutoff is the base model's.
- **No formal benchmark.** We validated it with hands-on smoke tests (security Q&A, tool calls, image reading), not a scored eval. Abliteration can slightly affect general output quality.
- **No safety net.** It will not refuse harmful requests, so your harness, policies, and human reviewers are the guardrails.
- **9B, not frontier.** It can hallucinate, misformat tool calls under ambiguity, and make factual errors. Verify commands before you run them.
- **Not a sandbox.** It emits tool *calls* as text; it does not execute anything. Wiring it to real tools is your responsibility, with your own authorization checks.

## License & attribution

Released under **Apache-2.0**. Derived from `Qwen/Qwen3.5-9B` (© Alibaba Cloud, Apache-2.0); see `NOTICE`. Made with [Heretic](https://github.com/p-e-w/heretic). You must retain the license and attribution in redistributions.

*Authorized-use-only for security work. Use responsibly and within the law.*