Text Generation
GGUF
Safetensors
English
cybersecurity
cve
cwe
cvss
tool-use
agent
lora
qwen3
conversational
Instructions to use dorb/ab9b-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use dorb/ab9b-v2 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf dorb/ab9b-v2:Q4_K_M # Run inference directly in the terminal: llama cli -hf dorb/ab9b-v2:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf dorb/ab9b-v2:Q4_K_M # Run inference directly in the terminal: llama cli -hf dorb/ab9b-v2:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf dorb/ab9b-v2:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf dorb/ab9b-v2:Q4_K_M
Use Docker
docker model run hf.co/dorb/ab9b-v2:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use dorb/ab9b-v2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "dorb/ab9b-v2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dorb/ab9b-v2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/dorb/ab9b-v2:Q4_K_M
- Ollama
How to use dorb/ab9b-v2 with Ollama:
ollama run hf.co/dorb/ab9b-v2:Q4_K_M
- Unsloth Desktop
- Pi
How to use dorb/ab9b-v2 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf dorb/ab9b-v2:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "dorb/ab9b-v2:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use dorb/ab9b-v2 with Docker Model Runner:
docker model run hf.co/dorb/ab9b-v2:Q4_K_M
- Lemonade
How to use dorb/ab9b-v2 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull dorb/ab9b-v2:Q4_K_M
Run and chat with the model
lemonade run user.ab9b-v2-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use dorb/ab9b-v2 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf dorb/ab9b-v2:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default dorb/ab9b-v2:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use dorb/ab9b-v2 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf dorb/ab9b-v2:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "dorb/ab9b-v2:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from dorb/ab9b-v2: direct link, hf CLI and curl.
- Browser
- Download file 4.89 kB
-
https://huggingface.co/dorb/ab9b-v2/resolve/main/README.md
- Command line
-
hf download hf://dorb/ab9b-v2/README.md
-
curl -L -o README.md https://huggingface.co/dorb/ab9b-v2/resolve/main/README.md
4.89 kB
| license: apache-2.0 | |
| base_model: Qwen/Qwen3.5-9B | |
| base_model_relation: finetune | |
| library_name: gguf | |
| pipeline_tag: text-generation | |
| language: | |
| - en | |
| tags: | |
| - cybersecurity | |
| - cve | |
| - cwe | |
| - cvss | |
| - tool-use | |
| - agent | |
| - lora | |
| - gguf | |
| - qwen3 | |
| # ab9b-v2 | |
| Defensive-cyber model on Qwen3.5-9B. Merged and quantised to Q4_K_M. Run the GGUF. The LoRA is here if you want the bf16 base plus the adapter instead. | |
| Trained for CVE→CWE, CVSS vectors, threat-intel questions, vulnerability review, and blue-team / DFIR / audit loops that call tools. Answers are short and lead with the conclusion. Thinking is off. | |
| ## Run | |
| ```bash | |
| huggingface-cli download dorb/ab9b-v2 ab9b-v2-Q4_K_M.gguf --local-dir . | |
| llama-server -m ab9b-v2-Q4_K_M.gguf -ngl 999 -fa on --jinja \ | |
| -c 32768 -np 1 --chat-template-kwargs '{"enable_thinking":false}' | |
| ``` | |
| OpenAI `tools` work with `--jinja`. The text protocol it was trained on is `CALL: tool_name(arg="…")` and then `FINAL: …`. | |
| The LoRA that was merged into that GGUF: | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| from peft import PeftModel | |
| base = "Qwen/Qwen3.5-9B" | |
| model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto") | |
| model = PeftModel.from_pretrained(model, "dorb/ab9b-v2", subfolder="adapters/v2a-ck900") | |
| tokenizer = AutoTokenizer.from_pretrained(base) | |
| ``` | |
| `adapters/v2b-final` and `adapters/v2c-final` are the later rounds. They were scored on the dev tier and not selected. Details are in [`evidence/REPORT.md`](evidence/REPORT.md). | |
| ## Test tier | |
| One scoring pass. Reference is `ab9b-r1` on the same base. Paired bootstrap, 95% CI. | |
| | | ab9b-v2 | ab9b-r1 | base | | |
| |---|---:|---:|---:| | |
| | cyber composite, 7 families | 0.739 | 0.725 | 0.719 | | |
| | CVE → CWE (n = 329) | 0.763 | 0.608 | | | |
| | CVSS vector (n = 334) | 0.812 | 0.715 | | | |
| | native tool-call loop, solved (n = 87) | 0.483 | 0.172 | 0.157 | | |
| | plain `CALL:` loop, solved (n = 89) | 0.528 | 0.483 | 0.236 | | |
| | single-call tool exact (n = 39) | 0.897 | 0.923 | 0.949 | | |
| | median seconds per answer | 1.89 | 3.68 | | | |
| | completion tokens per answer | 219 | 377 | | | |
| The composite move is +0.014 [−0.006, +0.033]. That misses the pre-registered +0.05 bar. The gains that clear their intervals are CVE→CWE (+0.155), CVSS (+0.097), and the native tool-call loop (+0.310). Median answer time is about half. Token count is about half. Decode speed is the same (~110 tok/s). | |
| On the same test items, against the other models that finished: wins over Cyber-Ornith-9B, Foundation-Sec-8B-Instruct, Primus-Merged-8B, SecGPT-7B, and the Qwen3.5-9B base. Ties with `ab9b-r1` and with Huihui-Qwen3.8-27B-abliterated. The 27B tie is at roughly twice the speed per quality point. Full table: [`evidence/results/league_verdict.txt`](evidence/results/league_verdict.txt). | |
| ## Training | |
| One LoRA round continued from `ab9b-r1`. r = 32, α = 32, lr 1e-4 cosine, one epoch, 1,063 steps, 16.06M tokens, 4.2 hours on one RTX 3090 (24 GB). Data is defensive: NVD CVE classification, vulnerability review, threat-intel Q&A, ATT&CK / CAPEC / CWE, analyst-brief rewrites, and oracle-checked agent episodes. Nothing from the reserved eval families. | |
| The shipped checkpoint is `v2a-ck900`. Two later rounds and two weight interpolations lost the pre-registered dev rule. `v2b` raised the agent scores and broke tool-call hygiene (it invents missing arguments). `v2c` repaired that hygiene and still scored below `v2a-ck900`, so it was never test-scored. Both adapters are in the repo so those tables can be checked. | |
| Protocol: [`evidence/GOAL.md`](evidence/GOAL.md). Numbers: [`evidence/REPORT.md`](evidence/REPORT.md). | |
| ## Files | |
| ``` | |
| ab9b-v2-Q4_K_M.gguf merged Q4_K_M, 5.8 GB | |
| adapters/v2a-ck900/ LoRA merged into the GGUF | |
| adapters/v2b-final/ later round, not selected | |
| adapters/v2c-final/ later round, not selected | |
| evidence/GOAL.md protocol, written before the test pass | |
| evidence/REPORT.md test table, league, rejected rounds | |
| evidence/results/ raw pick and verdict files | |
| SHA256SUMS GGUF and the three adapter weights | |
| ``` | |
| ## Limits | |
| Agent splits are small (89 / 87 / 39). Dev selection used 60 agent items, so a few points of noise is normal. Training families and the agent test share generators; eight reserved families were never trained on. CVE rows are the same task type as CTI-RCM and CTI-VSP, other CVEs, not the same items. Thinking mode, context past 64k, and languages other than English were not measured. The data is synthetic and defensive. It was not red-teamed. | |
| ## License | |
| Apache-2.0, same as Qwen3.5-9B. Public sources in the mix keep their own licences. | |
| ```bibtex | |
| @software{ab9b_v2_2026, | |
| title = {ab9b-v2}, | |
| author = {dorb}, | |
| year = {2026}, | |
| url = {https://huggingface.co/dorb/ab9b-v2} | |
| } | |
| ``` | |