CyberBuddy-CyberOps-9B

An uncensored cyber-operations assistant for authorized red- and blue-team work: it explains attacks, plans scans, and calls tools, without the base model's refusals.

CyberBuddy-CyberOps-9B is an abliterated Qwen/Qwen3.5-9B: the model's internal "refusal direction" was removed from its weights with Heretic. It is built for authorized security work where the base model's safety refusals get in the way: explaining how an exploit works, drafting a scan plan, reasoning about attacker tradecraft for defense, and triage. It calls tools in Qwen3.5's native XML tool-call format, and it can read images (screenshots, diagrams, charts).

⚠️ This model does not refuse. It will produce content the base model declines. Use it only for lawful, authorized work (pentests with permission, CTFs, research on systems you own or may test, defensive analysis), and put it behind a human-in-the-loop harness for any real-world action.

Formerly CyberBuddyAI/CyberOps-Qwen3.5-9B; old links redirect here.

Pick your version

Everything is in this one repo:

you want to…	use	GPU
run it on an 8 GB GPU (llama.cpp, LM Studio, Ollama…)	CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf, custom GGUF, 5.5 GiB	8 GB (64k context)
add image input to the GGUF	+ mmproj-CyberBuddy-CyberOps-9B-f16.gguf (0.9 GB)	~1 GB more
load it with 🤗 transformers, full quality	model-*.safetensors (bf16)	24 GB
load it with transformers in 4-bit	same files + load_in_4bit	10–12 GB

There is no LoRA adapter: abliteration edits the base weights directly. Nothing was trained, so there is nothing to apply on top of the base.

Full instructions: Deploy.

Highlights

• 🔓 No refusals on security work. Exploits, payload logic, recon plans, and attacker tradecraft are answered directly instead of blocked.
• 🧠 Works thinking OFF or ON. The refusals are gone in both modes — run it thinking-OFF for fast, concise tool-use, or thinking-ON for step-by-step reasoning. Neither mode refuses (see How it was made for why that is not trivial here).
• 🧰 Tool use. Emits well-formed Qwen3.5-native <tool_call>s; served with llama.cpp --jinja you get parsed OpenAI tool_calls.
• 👁️ Sees images. The Qwen3.5 vision encoder is kept, byte-identical to the base model; ship the mmproj file with the GGUF for image input.
• 🪶 Runs small. The custom GGUF (4-bit layers, 8-bit embeddings) fits on an 8 GB GPU with 64k context.
• 🎯 Minimal edit. Only 60 of 426 language-model weight tensors changed; the base model's knowledge and skills are kept.

Intended use & responsible use

• Use for: authorized penetration testing, CTF competitions, red/blue-team exercises, security research on systems you own or are permitted to test, and defensive analysis of attacker behavior.
• Users: security professionals and researchers working inside an authorized scope.
• Not for: unauthorized access, harm to systems or people, or any unlawful activity. Removing refusals is a tradeoff made so professionals can use the model inside their scope; it is not an endorsement of misuse. You are responsible for how you use it.
• Deploy it with a human in the loop. Let the model propose actions, and have a person approve anything that touches a real system.

How to use

The model uses the stock Qwen3.5 chat template and works with thinking either OFF or ON — refusals are removed in both modes. It was validated in thinking-OFF mode (enable_thinking=False), the recommended default for tool-use: concise, direct answers. Thinking-ON also works and is not refused — see Generation settings for the one setting that matters (a large token budget). Pass your tools with apply_chat_template(..., tools=...); the template renders them in the model's own format.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

mid = "CyberBuddyAI/CyberBuddy-CyberOps-9B"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype=torch.bfloat16, device_map="auto")

tools = [{"type": "function", "function": {
    "name": "nmap_scan",
    "description": "Run an authorized nmap scan.",
    "parameters": {"type": "object", "properties": {
        "target": {"type": "string"}, "ports": {"type": "string"}}, "required": ["target"]}}}]
messages = [
    {"role": "system", "content": "You are a security operations assistant. Use tools when useful."},
    {"role": "user", "content": "Run a TCP port scan of 10.0.0.5, ports 1-1024."},
]
inputs = tok.apply_chat_template(messages, tools=tools, add_generation_prompt=True, enable_thinking=False,
                                 return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False,
                     stop_strings=["</tool_call>"], tokenizer=tok)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

AutoModelForCausalLM loads the text model only (8.95B params) and skips the vision encoder. For images in transformers, use AutoProcessor + AutoModelForImageTextToText the same way as for base Qwen3.5 (needs pillow and torchvision). We tested image input through llama.cpp (below), not through transformers.

💡 Agent loop: stop at the first </tool_call>, run the tool, append the result as a {"role": "tool", ...} message, and generate again.

Deploy

1. Pick a build for your GPU

The text GGUF has the same architecture and the exact same size (5,875,442,272 bytes) as our CyberBuddy-CloudSec-9B build. The VRAM numbers below were measured on that build (RTX 5070 Ti Laptop GPU, nvidia-smi, weights + KV cache + buffers), text only.

your GPU	build	settings	VRAM used
6 GB	GGUF	-c 4096 … 16384	4.9 GiB at 4k
8 GB	GGUF	-c 65536	6.9 GiB at 64k
8 GB, long context	GGUF	-c 131072 -ctk q8_0 -ctv q8_0 (8-bit KV cache)	7.5 GiB at 128k (tight)
10–12 GB	GGUF	-c 131072 (f16 KV cache)	8.9 GiB at 128k
10–12 GB	this repo, transformers + bitsandbytes 4-bit	see below	~7.1 GiB weights at short context
24 GB+	this repo, transformers bf16	see How to use	~17 GB weights (text)

With images: loading the mmproj projector alongside the GGUF uses ~7.1 GiB (measured, -c 32768, idle), plus a little working memory per image. It fits an 8 GB GPU at a shorter context like -c 32768; for more headroom use 10–12 GB.

Why it stays small at long context: Qwen3.5-9B is a hybrid. Only 8 of its 32 layers use full attention (with 4 KV heads); the other 24 use Gated DeltaNet, a linear-attention layer with a fixed-size state. So the KV cache grows only ~32 KB per token in f16. That is about 2 GB for 64k tokens, and half that with an 8-bit KV cache.

2. llama.cpp server (recommended for small GPUs)

Tested with llama.cpp release b12327.

Text only:

llama-server -hf CyberBuddyAI/CyberBuddy-CyberOps-9B --hf-file CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf \
  -ngl 99 -c 65536 -fa on --jinja --reasoning off --temp 0 --port 8080

Text + images: download both files, then pass the projector with --mmproj:

hf download CyberBuddyAI/CyberBuddy-CyberOps-9B \
  CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf mmproj-CyberBuddy-CyberOps-9B-f16.gguf --local-dir .
llama-server -m CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf --mmproj mmproj-CyberBuddy-CyberOps-9B-f16.gguf \
  -ngl 99 -c 32768 -fa on --jinja --reasoning off --temp 0 --port 8080

flag	what it does	why we set it
-ngl 99	put all layers on the GPU	the whole model fits; partial offload is much slower
-c 65536	context window (tokens)	the main VRAM knob: each 1k tokens costs ~32 MB (f16 KV)
-fa on	flash attention	less memory and faster; also needed for a quantized KV cache
-ctk q8_0 -ctv q8_0	store the KV cache in 8-bit	halves KV memory → 128k context on 8 GB. Optional
--jinja	use the model's own chat template	required for tool calling: the server then parses <tool_call> into OpenAI tool_calls
--reasoning off / --reasoning on	thinking OFF (default here) or ON	de-refused in both modes; off is the validated default. With --reasoning on, allow a long response so the <think> block has room
--mmproj FILE	load the vision projector	enables image input (OpenAI image_url content parts)
--temp 0	greedy decoding	deterministic answers for tool use

The server speaks the OpenAI API (/v1/chat/completions), so any OpenAI client can pass tools=[...] and images.

Example requests (real results from this GGUF, greedy):

# tool call
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [
    {"role": "system", "content": "You are a security operations assistant. Use tools when useful."},
    {"role": "user", "content": "Run a TCP port scan of 10.0.0.5, ports 1-1024."}],
  "tools": [{"type": "function", "function": {"name": "nmap_scan",
    "description": "Run an authorized nmap scan.",
    "parameters": {"type": "object", "properties": {"target": {"type": "string"}, "ports": {"type": "string"}},
                   "required": ["target"]}}}],
  "temperature": 0}'

→ parsed tool call nmap_scan({"target": "10.0.0.5", "ports": "1-1024"}).

• "How does ARP spoofing work on a LAN?" → a correct 3-step explanation: forged ARP replies, cache poisoning, man-in-the-middle.
• An image of two training-loss charts, "what happens to the val loss on the left?" → "the validation loss initially decreases but then begins to rise after around step 600, while the training loss continues to fall … the model is overfitting." (correct)

LM Studio / Ollama / other llama.cpp apps: download the GGUF (and the mmproj file for images) from this repo. It should work in any app whose bundled llama.cpp supports the Qwen3.5 architecture (qwen35); we have not tested these apps ourselves.

The custom GGUF quant

tensor	type	why
token embedding (token_embd, 248k × 4096)	Q8_0	the input embedding is ~1B params; 8-bit keeps every token's vector nearly exact
transformer layers	Q4_K_M mix (Q4_K, with Q6_K on the most sensitive tensors)	standard 4-bit k-quant
output head (output)	Q4_K	saves ~0.5 GB vs 8-bit, which buys more context on small GPUs
vision projector (mmproj)	F16	small (0.9 GB) and kept at full precision

Built with llama.cpp b12327:

llama-quantize --token-embedding-type q8_0 --output-tensor-type q4_k \
  CyberBuddy-CyberOps-9B-bf16.gguf CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf Q4_K_M

3. transformers, 4-bit (10–12 GB GPU)

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

mid = "CyberBuddyAI/CyberBuddy-CyberOps-9B"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, quantization_config=bnb, dtype=torch.bfloat16, device_map="auto")
# then use the same generate code as in "How to use"

• Speed tip: install flash-linear-attention and causal-conv1d. Without them, transformers falls back to a slow pure-PyTorch path for the Gated DeltaNet layers.
• Needs a recent transformers with Qwen3.5 support (we used 5.6.2).

4. vLLM

Validated with vLLM 0.22.1, 4-bit (bitsandbytes), on a single 12 GB GPU:

vllm serve CyberBuddyAI/CyberBuddy-CyberOps-9B \
    --quantization bitsandbytes --dtype bfloat16 --language-model-only \
    --max-model-len 12288 --max-num-seqs 16 --gpu-memory-utilization 0.90 \
    --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
    --default-chat-template-kwargs '{"enable_thinking": false}'

--tool-call-parser qwen3_coder matches the model's tool-call format (<tool_call><function=NAME><parameter=...>).

5. Generation settings

use	settings
tool-calling agents (recommended default)	greedy: do_sample=False / temperature=0, thinking OFF
open-ended answers, thinking OFF	Qwen's non-thinking defaults: temperature=0.7, top_p=0.8, top_k=20
thinking ON (step-by-step reasoning)	enable_thinking=True, temperature=0.6, top_p=0.95, large max_new_tokens (~4k) — the <think> block can be long; a small budget leaves no room for the answer

Files in this repo

path	what
model-*.safetensors, config.json, generation_config.json, tokenizer + chat template	full bf16 model (Qwen3_5ForConditionalGeneration: language model + vision encoder)
preprocessor_config.json, video_preprocessor_config.json	image/video processor settings (same as base Qwen3.5-9B)
CyberBuddy-CyberOps-9B-Q4_K_M-embQ8.gguf	custom GGUF for llama.cpp, text model (5.47 GiB, sha256 51ba64d9309a6c0566df097da898a7b72687721a8ef4b1f2e3e6b580267282f5)
mmproj-CyberBuddy-CyberOps-9B-f16.gguf	vision projector for llama.cpp image input (0.86 GiB, sha256 d55beb9dc4e47508c223d8c5a87952ccd800fa3f5bdc4dfc05b8df0f34cfd763)
LICENSE, NOTICE	Apache-2.0 + Qwen attribution

How it was made

Base	Qwen/Qwen3.5-9B (Apache-2.0, hybrid Gated DeltaNet + full attention, with vision encoder)
Method	abliteration (refusal-direction ablation) with Heretic, in thinking-OFF mode
Data	none. Abliteration finds the refusal direction by contrasting activations on harmful vs. harmless prompts, then removes it from the weights; no fine-tuning data is added
What changed	60 of 426 language-model weight tensors; the vision encoder (333 tensors) is byte-identical to the base

In plain terms: the model represents "I should refuse this" as a direction in its activation space. Heretic finds that direction and subtracts it from the weights, so the model stops steering toward refusals. Knowledge and skills come from the base model.

Why this one was hard — and the method that worked

A one-shot "find the refusal direction and subtract it" fails on this model, for two concrete reasons we measured:

• It starts saturated. In its target thinking-OFF deployment mode, the base model refuses 100 of 100 of our harmful probe prompts. A search that scores itself by how many refusals are left gets no signal while every trial still scores 100/100 — there is nothing to climb toward. A naive single-direction subtraction also plateaus, stuck well short of full de-refusal.
• The obvious shortcut is wrong. You might guess the refusal "lives in the chain-of-thought," so decensoring with thinking ON would be easier. We measured the opposite: thinking ON refuses 89/100, thinking OFF refuses 100/100 — thinking-off refuses more, not less. So we could not take the easy mode; we had to crack the harder, saturated one, because thinking-off is how the model is actually deployed.

What worked is Heretic's guided search instead of a one-shot edit:

1. Find the direction well. Contrast activations on harmful vs. harmless prompts to extract a refusal direction per layer (here it came out usable — cluster silhouette ≈ 0.33).
2. Search, don't guess. Run a 120-trial TPE (Optuna) optimization over the per-layer ablation strengths, scored on both refusals removed and a KL-divergence budget that caps how far the rest of the model is allowed to drift.
3. Ablate harder to break the plateau. We raised the ablation-weight ceiling (1.5 → 3.0) and widened the KL budget (0.01 → 0.15), so the search is rewarded for trading a little extra model-change for far fewer refusals. The winning trials sit at KL ≈ 0.01–0.05 — well below the ~0.5 level where a model starts to visibly break.

The result changes only 60 of 426 language-model weight tensors (the vision encoder is untouched — byte-identical to the base), and although it was tuned in thinking-OFF mode, the de-refusal carries over to thinking-ON too — what you'd expect from removing a single shared refusal direction.

Limitations

• Same knowledge as the base. Abliteration removes refusals; it does not add knowledge or skill, and the knowledge cutoff is the base model's.
• No formal benchmark. We validated it with hands-on smoke tests (security Q&A, tool calls, image reading), not a scored eval. Abliteration can slightly affect general output quality.
• No safety net. It will not refuse harmful requests, so your harness, policies, and human reviewers are the guardrails.
• 9B, not frontier. It can hallucinate, misformat tool calls under ambiguity, and make factual errors. Verify commands before you run them.
• Not a sandbox. It emits tool calls as text; it does not execute anything. Wiring it to real tools is your responsibility, with your own authorization checks.

License & attribution

Released under Apache-2.0. Derived from Qwen/Qwen3.5-9B (© Alibaba Cloud, Apache-2.0); see NOTICE. Made with Heretic. You must retain the license and attribution in redistributions.

Authorized-use-only for security work. Use responsibly and within the law.