ab9b-v2 / README.md
dorb's picture
Publish ab9b-v2
e647719 verified
|
Raw History Blame Contribute Delete
4.89 kB
---
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
base_model_relation: finetune
library_name: gguf
pipeline_tag: text-generation
language:
- en
tags:
- cybersecurity
- cve
- cwe
- cvss
- tool-use
- agent
- lora
- gguf
- qwen3
---
# ab9b-v2
Defensive-cyber model on Qwen3.5-9B. Merged and quantised to Q4_K_M. Run the GGUF. The LoRA is here if you want the bf16 base plus the adapter instead.
Trained for CVE→CWE, CVSS vectors, threat-intel questions, vulnerability review, and blue-team / DFIR / audit loops that call tools. Answers are short and lead with the conclusion. Thinking is off.
## Run
```bash
huggingface-cli download dorb/ab9b-v2 ab9b-v2-Q4_K_M.gguf --local-dir .
llama-server -m ab9b-v2-Q4_K_M.gguf -ngl 999 -fa on --jinja \
-c 32768 -np 1 --chat-template-kwargs '{"enable_thinking":false}'
```
OpenAI `tools` work with `--jinja`. The text protocol it was trained on is `CALL: tool_name(arg="…")` and then `FINAL: …`.
The LoRA that was merged into that GGUF:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = "Qwen/Qwen3.5-9B"
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "dorb/ab9b-v2", subfolder="adapters/v2a-ck900")
tokenizer = AutoTokenizer.from_pretrained(base)
```
`adapters/v2b-final` and `adapters/v2c-final` are the later rounds. They were scored on the dev tier and not selected. Details are in [`evidence/REPORT.md`](evidence/REPORT.md).
## Test tier
One scoring pass. Reference is `ab9b-r1` on the same base. Paired bootstrap, 95% CI.
| | ab9b-v2 | ab9b-r1 | base |
|---|---:|---:|---:|
| cyber composite, 7 families | 0.739 | 0.725 | 0.719 |
| CVE → CWE (n = 329) | 0.763 | 0.608 | |
| CVSS vector (n = 334) | 0.812 | 0.715 | |
| native tool-call loop, solved (n = 87) | 0.483 | 0.172 | 0.157 |
| plain `CALL:` loop, solved (n = 89) | 0.528 | 0.483 | 0.236 |
| single-call tool exact (n = 39) | 0.897 | 0.923 | 0.949 |
| median seconds per answer | 1.89 | 3.68 | |
| completion tokens per answer | 219 | 377 | |
The composite move is +0.014 [−0.006, +0.033]. That misses the pre-registered +0.05 bar. The gains that clear their intervals are CVE→CWE (+0.155), CVSS (+0.097), and the native tool-call loop (+0.310). Median answer time is about half. Token count is about half. Decode speed is the same (~110 tok/s).
On the same test items, against the other models that finished: wins over Cyber-Ornith-9B, Foundation-Sec-8B-Instruct, Primus-Merged-8B, SecGPT-7B, and the Qwen3.5-9B base. Ties with `ab9b-r1` and with Huihui-Qwen3.8-27B-abliterated. The 27B tie is at roughly twice the speed per quality point. Full table: [`evidence/results/league_verdict.txt`](evidence/results/league_verdict.txt).
## Training
One LoRA round continued from `ab9b-r1`. r = 32, α = 32, lr 1e-4 cosine, one epoch, 1,063 steps, 16.06M tokens, 4.2 hours on one RTX 3090 (24 GB). Data is defensive: NVD CVE classification, vulnerability review, threat-intel Q&A, ATT&CK / CAPEC / CWE, analyst-brief rewrites, and oracle-checked agent episodes. Nothing from the reserved eval families.
The shipped checkpoint is `v2a-ck900`. Two later rounds and two weight interpolations lost the pre-registered dev rule. `v2b` raised the agent scores and broke tool-call hygiene (it invents missing arguments). `v2c` repaired that hygiene and still scored below `v2a-ck900`, so it was never test-scored. Both adapters are in the repo so those tables can be checked.
Protocol: [`evidence/GOAL.md`](evidence/GOAL.md). Numbers: [`evidence/REPORT.md`](evidence/REPORT.md).
## Files
```
ab9b-v2-Q4_K_M.gguf merged Q4_K_M, 5.8 GB
adapters/v2a-ck900/ LoRA merged into the GGUF
adapters/v2b-final/ later round, not selected
adapters/v2c-final/ later round, not selected
evidence/GOAL.md protocol, written before the test pass
evidence/REPORT.md test table, league, rejected rounds
evidence/results/ raw pick and verdict files
SHA256SUMS GGUF and the three adapter weights
```
## Limits
Agent splits are small (89 / 87 / 39). Dev selection used 60 agent items, so a few points of noise is normal. Training families and the agent test share generators; eight reserved families were never trained on. CVE rows are the same task type as CTI-RCM and CTI-VSP, other CVEs, not the same items. Thinking mode, context past 64k, and languages other than English were not measured. The data is synthetic and defensive. It was not red-teamed.
## License
Apache-2.0, same as Qwen3.5-9B. Public sources in the mix keep their own licences.
```bibtex
@software{ab9b_v2_2026,
title = {ab9b-v2},
author = {dorb},
year = {2026},
url = {https://huggingface.co/dorb/ab9b-v2}
}
```